office/architecture.html

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <title>Architecture — Nextcloud Office</title>
  <link rel="stylesheet" href="assets/style.css">
</head>
<body>
<div id="wrapper">

  <div class="crumbs"><a href="index.html">Nextcloud Office</a> &nbsp;›&nbsp; Architecture</div>

  <ul class="nav">
    <li><a href="index.html">Overview</a></li>
    <li><a href="architecture.html" class="active">Architecture</a></li>
    <li><a href="procedure.html">Procedure</a></li>
    <li><a href="troubleshooting.html">Troubleshooting</a></li>
    <li><a href="operations.html">Operations</a></li>
  </ul>

  <h1>Architecture</h1>
  <p>
    Three layers to understand: <strong>network</strong> (public → Caddy →
    NetBird → AIO Apache), <strong>containers</strong> (8 AIO processes on
    one host, host network namespace shared with mastercontainer), and
    <strong>data flow</strong> (NFS for user files, local ext4 for
    everything else, plus a daily postgres dump that lands on NFS).
  </p>

  <h2>Topology — public ingress</h2>
  <p>
    Three active layers, plus the retired OnlyOffice tier that's been
    torn down:
  </p>
  <p><img src="assets/diagrams/topology.svg" alt="Topology — Caddy on hawker, NetBird mesh, AIO on homework03, NFS to desslok" class="diagram"></p>

  <ol>
    <li>
      <strong>Public ingress</strong> — Cloudflare DNS points
      <code>office.rmf44.xyz</code> directly at <code>hawker
      (192.255.159.202)</code>. Caddy in the <code>caddy-caddy-1</code>
      container terminates TLS with a Let's Encrypt cert and
      reverse-proxies to <code>100.79.142.164:11000</code>
      (homework03's NetBird IP, AIO Apache port).
    </li>
    <li>
      <strong>NetBird mesh</strong> — WireGuard P2P between hawker
      (<code>100.79.4.103</code>) and homework03
      (<code>100.79.142.164</code>). Direct host-host (no relay) over
      UDP 51820.
    </li>
    <li>
      <strong>Compute + data</strong> — 8 AIO containers on homework03,
      sharing host network via <code>network_mode: host</code> on the
      mastercontainer. Apache listens on host :11000; mastercontainer
      owns :80, :8080, :8443, :9000.
    </li>
    <li>
      <strong>Storage</strong> — NFSv4.1 from
      <code>desslok:/slab/container_storage/office</code> mounted at
      <code>/srv/nc-files</code> on homework03. User files live at
      the NFS export root (no intermediate <code>nextcloud/</code>
      subdir): <code>admin/</code>, <code>race/</code>,
      <code>appdata_*/</code>, plus <code>backups/</code> for the
      daily backup pipeline.
    </li>
    <li>
      <strong>Retired (torn down)</strong> — OnlyOffice container
      (<code>onlyoffice-files-1</code>) on hawker, plus its nginx
      vhost, plus its daily backup pipeline on hector. Replaced by
      the AIO stack.
    </li>
  </ol>

  <h2>Container tree — what's running on homework03</h2>
  <p>
    AIO manages its own container lifecycle; you start the
    <em>mastercontainer</em> via <code>docker compose up -d</code> and
    it spawns the rest. Each side container has a named docker volume
    bound to a host directory so the data survives mastercontainer
    restarts.
  </p>
  <p><img src="assets/diagrams/container-tree.svg" alt="AIO container tree with bind mounts and ports" class="diagram"></p>

  <table>
    <tr><th>Container</th><th>Role</th><th>Bind target</th><th>Owner</th><th>Port</th></tr>
    <tr>
      <td><code>nextcloud-aio-mastercontainer</code></td>
      <td>Orchestrator, domain validator, admin UI</td>
      <td><code>./nextcloud-aio-mastercontainer/</code></td>
      <td><code>33:33</code> (www-data)</td>
      <td>:80 (acme), :8080 (admin UI), :8443 (alt admin), :9000 (nextcloud-fcgi via apache)</td>
    </tr>
    <tr>
      <td><code>nextcloud-aio-apache</code></td>
      <td>Reverse proxy → nextcloud-fcgi, public-facing</td>
      <td><code>./nextcloud-aio-apache/</code></td>
      <td><code>33:33</code></td>
      <td>:11000 (host) → :11000 (container)</td>
    </tr>
    <tr>
      <td><code>nextcloud-aio-nextcloud</code></td>
      <td>PHP-FPM + Nextcloud app code</td>
      <td><code>./nextcloud-aio-nextcloud/</code> (local volume) + <code>/srv/nc-files</code> (NFS) → <code>/mnt/ncdata</code></td>
      <td>root (entrypoint)</td>
      <td>:9000 (PHP-FPM)</td>
    </tr>
    <tr>
      <td><code>nextcloud-aio-database</code></td>
      <td>PostgreSQL 16</td>
      <td><code>./nextcloud-aio-database/</code></td>
      <td><code>999:999</code></td>
      <td>:5432 (internal only)</td>
    </tr>
    <tr>
      <td><code>nextcloud-aio-redis</code></td>
      <td>Cache + file locking</td>
      <td><code>./nextcloud-aio-redis/</code></td>
      <td><code>999:999</code></td>
      <td>:6379 (internal only)</td>
    </tr>
    <tr>
      <td><code>nextcloud-aio-collabora</code></td>
      <td>CODE Office (Word/Excel/PowerPoint editing)</td>
      <td><code>./nextcloud-aio-collabora/</code></td>
      <td><code>100:101</code></td>
      <td>:9980 (internal only, called by apache)</td>
    </tr>
    <tr>
      <td><code>nextcloud-aio-whiteboard</code></td>
      <td>Built-in collaborative whiteboard</td>
      <td><code>./nextcloud-aio-whiteboard/</code></td>
      <td>(n/a)</td>
      <td>:3002 (internal only)</td>
    </tr>
    <tr>
      <td><code>nextcloud-aio-notify-push</code></td>
      <td>Push notification backend (websocket)</td>
      <td><code>./nextcloud-aio-notify-push/</code></td>
      <td>(n/a)</td>
      <td>:7867 (internal only)</td>
    </tr>
    <tr>
      <td><code>nextcloud-aio-imaginary</code></td>
      <td>(DISABLED — saves RAM)</td>
      <td>—</td>
      <td>—</td>
      <td>—</td>
    </tr>
    <tr>
      <td><code>nextcloud-aio-fulltextsearch</code></td>
      <td>(DISABLED — saves RAM)</td>
      <td>—</td>
      <td>—</td>
      <td>—</td>
    </tr>
    <tr>
      <td><code>nextcloud-aio-clamav</code></td>
      <td>(DISABLED — saves RAM)</td>
      <td>—</td>
      <td>—</td>
    </tr>
  </table>

  <div class="callout info">
    <p>
      <strong>Why host network?</strong> The AIO mastercontainer runs
      with <code>network_mode: host</code> so it can publish ports :80
      and :8443 directly on the host's network namespace. Apache (the
      sidecar) is reached via host :11000 because AIO's domain
      validation flow requires mastercontainer own host :80.
    </p>
  </div>

  <h2>Data flow — where each piece lives</h2>
  <p>
    Most of AIO's user data is on NFS
    (<code>/srv/nc-files</code> on homework03 → container bind at
    <code>/mnt/ncdata</code>). Postgres, Redis, and the AIO
    mastercontainer's configuration live on local ext4 (named
    volumes) — keeping PostgreSQL's WAL writes off NFS is critical
    for durability.
  </p>
  <p><img src="assets/diagrams/data-flow.svg" alt="Data flow — NFS for user files, named volumes for container state" class="diagram"></p>

  <table>
    <tr><th>Path</th><th>Filesystem</th><th>Why</th></tr>
    <tr>
      <td><code>/srv/nc-files/</code> (NFS root on homework03)</td>
      <td>NFSv4.1 from desslok</td>
      <td>User-uploaded files live at the export root: <code>admin/</code>, <code>race/</code>, <code>appdata_*/</code>, etc. The nextcloud container binds this path to <code>/mnt/ncdata</code> directly. Snapshotted daily via desslok's ZFS path.</td>
    </tr>
    <tr>
      <td><code>/srv/nc-files/backups/</code></td>
      <td>NFSv4.1 from desslok</td>
      <td>Daily pgdump + AIO config tar + user-files tar. 14-day retention.</td>
    </tr>
    <tr>
      <td><code>/usr/local/containers/nextcloudaio/nextcloud-aio-nextcloud/_data/</code></td>
      <td>ext4 (local)</td>
      <td>Local named-volume bind for the nextcloud container. AIO-managed; <code>NEXTCLOUD_DATADIR</code> points at <code>/srv/nc-files</code> separately, NOT at this volume.</td>
    </tr>
    <tr>
      <td><code>/usr/local/containers/nextcloudaio/nextcloud-aio-{mastercontainer,database,redis,apache,...}/</code></td>
      <td>ext4 (local)</td>
      <td>Named-volume bind targets per container. Each holds the writable state for that one container.</td>
    </tr>
  </table>

  <div class="callout info">
    <p>
      <strong>Rewire 2026-08-11:</strong> before the rewire, AIO
      bind-mounted a <em>third</em> host layer
      (<code>/mnt/nc-data/nextcloud-data</code>) into the nextcloud
      container, with <code>NEXTCLOUD_DATADIR=/mnt/nc-data/nextcloud-data</code>.
      That double-bind worked but had two sharp edges: (a) the
      <code>NEXTCLOUD_DATADIR</code> host path and the actual NFS
      mount path had to be kept in sync manually; (b) a typo would
      silently create an empty datadir, masking the real NFS data
      and triggering an "Appdata is not present" crash on next
      start. The rewire collapses both to a single layer:
      <code>/srv/nc-files</code> is the NFS mount AND the
      <code>NEXTCLOUD_DATADIR</code> value — AIO binds it directly
      into the nextcloud container at <code>/mnt/ncdata</code>.
    </p>
  </div>

  <div class="callout warn">
    <p>
      <strong>Why isn't the postgres data on NFS?</strong>
      PostgreSQL's WAL writes are sensitive to NFS close-to-open
      consistency. The official AIO image puts the database on a local
      named volume by default and we kept that. <code>pg_dumpall</code>
      (which is what the backup pipeline runs) produces a
      crash-consistent snapshot at dump time, so the daily backup is
      good — but live writes from postgres go to local ext4 only.
    </p>
  </div>

  <h2>Request flow — file upload via WebDAV</h2>
  <p>
    Swimlane sequence diagram of a single file upload:
    <code>curl -T smoke-test.md https://office.rmf44.xyz/remote.php/dav/files/admin/smoke-test.md</code>
    as run during the smoke test on 2026-08-10:
  </p>
  <p><img src="assets/diagrams/request-flow.svg" alt="Request flow — curl PUT through Caddy, NetBird, AIO Apache, PHP-FPM, NFS" class="diagram"></p>

  <ol>
    <li><strong>DNS</strong> — <code>office.rmf44.xyz</code> resolves to <code>192.255.159.202</code> (hawker).</li>
    <li><strong>TLS handshake</strong> — Caddy presents the Let's Encrypt cert for <code>office.rmf44.xyz</code>.</li>
    <li><strong>HTTPS PUT</strong> arrives at hawker on :443 with path <code>/remote.php/dav/files/admin/smoke-test.md</code>.</li>
    <li><strong>Caddy route</strong> matches the <code>Host: office.rmf44.xyz</code> block and forwards to <code>100.79.142.164:11000</code>.</li>
    <li><strong>NetBird tunnel</strong> encapsulates the request in WireGuard (UDP 51820), P2P from hawker to homework03.</li>
    <li><strong>AIO Apache</strong> (nextcloud-aio-apache container) terminates the TLS-stripped HTTP and forwards to <code>127.0.0.1:9000</code> (PHP-FPM in nextcloud-aio-nextcloud).</li>
    <li><strong>PHP-FPM (Nextcloud)</strong> authenticates the user via the session cookie, authorizes the path under <code>/admin/files/</code>, and writes the file via WebDAV.</li>
    <li><strong>NFS write</strong> of the file to <code>/srv/nc-files/admin/files/smoke-test.md</code> on homework03 → <code>/slab/container_storage/office/admin/files/smoke-test.md</code> on desslok (nextcloud container's <code>/mnt/ncdata</code> is bound to <code>/srv/nc-files</code> directly).</li>
    <li><strong>Response</strong>: <code>HTTP/2 201 Created</code> with empty body (WebDAV semantics).</li>
  </ol>

  <h2>Collabora editing flow</h2>
  <p>
    When a user opens a .docx in the web UI, Nextcloud embeds
    Collabora in an iframe via WOPI. The full chain:
  </p>
  <ol>
    <li>User clicks "Open in Collabora" in the Nextcloud file UI.</li>
    <li>Nextcloud generates a one-time WOPI token for the file.</li>
    <li>Iframe loads <code>https://office.rmf44.xyz/apps/richdocuments/index?fileId=123&amp;requesttoken=...</code>.</li>
    <li>Browser fetches the iframe content from Caddy → Apache → PHP-FPM.</li>
    <li>PHP-FPM serves the richdocuments app HTML.</li>
    <li>Browser opens a second HTTPS connection to <code>https://office.rmf44.xyz:9980</code>... no, actually: AIO proxies Collabora internally at <code>http://nextcloud-aio-apache.nextcloud-aio:23973</code>. The browser never sees Collabora directly.</li>
    <li>Collabora loads the file via WOPI (read from Nextcloud's WebDAV), serves the editor in the iframe, autosaves back through WOPI.</li>
  </ol>

  <div class="callout info">
    <p>
      <strong>Verified:</strong> <code>docker exec nextcloud-aio-nextcloud
      sudo -u www-data php occ config:app:get richdocuments wopi_url</code>
      returned <code>http://nextcloud-aio-apache.nextcloud-aio:23973</code>
      — internal network address, not exposed publicly. The WOPI secret
      never leaves the docker network.
    </p>
  </div>

</div>
</body>
</html>