CelinQ Insights · No. 40

Enterprise Architect Performance Benchmark: Local Repository, Remote Repository and CelinQ

Before you believe a single performance claim about Enterprise Architect over a network — ours included — ask to see the method behind it, not just the number.

Every vendor in this space eventually says some version of the same sentence: our approach makes Enterprise Architect faster over a network. It is said in sales decks, in one-page comparison sheets, occasionally in a case study with a client name attached and occasionally without one. It is almost never accompanied by the thing that would let a sceptical architect actually check it: the test hardware, the Enterprise Architect version, the repository that was used, the network conditions that were simulated, and the exact operation that was timed. A number without that context is not evidence. It is a marketing sentence wearing a lab coat, and the discipline that separates it from an actual finding is not complicated. It is just rarely done, because doing it properly is more work than writing the sentence, and because a properly documented benchmark exposes exactly where a product is strong and exactly where it is not, which is a less comfortable thing to publish than a single flattering multiple.

This article does not give you a table of milliseconds. That is deliberate, and it is worth explaining why before going further, because the absence is the point rather than an oversight. CelinQ has not published a controlled, repeatable benchmark of raw Enterprise Architect operation latency — element creation, rename, diagram open, search — across a local repository, a traditional remote repository, and a CelinQ-synchronised repository, run on fixed hardware against fixed simulated network conditions. Until that specific benchmark exists, with its method published alongside its numbers, printing a number here would be exactly the practice this article is arguing against. What follows instead is the methodology: what such a benchmark has to measure, how to construct it honestly, and why the shape of the test matters as much as the result. If you take nothing else from this, take the checklist, and use it on every vendor who tells you their product is fast, including this one.

What "benchmark" should mean for a modelling tool

A benchmark earns the word when a stranger can take the description of it and reproduce something close to the same result without access to the original author. That is a higher bar than it sounds. It rules out "our internal testing showed," because internal testing that nobody else can rerun is a claim, not a benchmark. It rules out an average taken from a handful of anecdotal timings on someone's laptop during a demo, because a demo is optimised, consciously or not, for the moment it is being shown, and a single run tells you almost nothing about variance. It also rules out comparing two things that were not actually put through the same test — timing your own product on a fast local network and quoting a competitor's number from their own marketing material, measured under conditions nobody can verify, is not a comparison at all, however tempting the juxtaposition is to put on a slide.

A benchmark that deserves the name publishes, at minimum, the exact hardware the test ran on — CPU, memory, storage type, because a spinning disk and an NVMe drive produce meaningfully different local I/O costs that have nothing to do with the network story being told. It publishes the exact Enterprise Architect version, because EA's own internal behaviour has changed across major releases and a result from one version does not automatically transfer to another. It publishes the size and shape of the repository under test — element count, package depth, diagram count and diagram complexity — because a hundred-element demo repository and a fifty-thousand-element production repository do not behave the same way even on identical hardware, and a vendor who benchmarks on the small one while selling to organisations with the large one is not lying exactly, but is not being straight either. It publishes the network conditions, precisely, in milliseconds of round-trip time and any bandwidth cap applied, rather than a vague label like "simulated WAN." And it publishes the exact operation being timed, because "opening a diagram" and "creating an element" and "running a search across the model" are different operations with different cost profiles, and collapsing them into a single average latency number hides more than it reveals.

A number is not evidence until you can see the test that produced it. Ask for the hardware, the EA version, the repository size, the network conditions, and the exact operation timed — and be as suspicious of a vendor who won't give you those as you would be of a scientific paper with no methods section.

Isolating latency from bandwidth

The single most common confusion in casual performance discussion about Enterprise Architect over a network is treating "slow" as one undifferentiated thing, when in practice it is usually one of two quite different problems that happen to produce a similar symptom. Bandwidth is how much data can move per second once a connection is established; latency is how long a single round trip takes regardless of how much data is in it. A connection with generous bandwidth and poor latency will still feel sluggish for a tool like Enterprise Architect, because so much of what EA does against a remote repository is not moving large amounts of data — it is making many small, sequential requests, each of which has to complete before the next one can sensibly begin. Opening a package, populating a tree, reading the properties of an element you just clicked: these are not big transfers. They are conversations, and a conversation with a long pause after every sentence is exhausting even if each sentence itself is short.

This is why a methodologically serious benchmark isolates round-trip time as its primary independent variable rather than lumping "network quality" into one dial. A useful test harness holds bandwidth generous and fixed, then steps latency through a sequence of realistic values — commonly something like 20, 50, 100 and 200 milliseconds of added round-trip time, chosen because they map onto recognisable real situations: 20 milliseconds resembles a well-connected office on a metropolitan link, 50 resembles a reasonable inter-city or cross-border connection, 100 resembles a longer-haul or heavily-routed VPN path, and 200 resembles what an architect on the far side of a satellite link, a poorly peered international connection, or a congested VPN concentrator might actually experience on an ordinary Tuesday. The point of stepping through several values rather than testing one is that it reveals the shape of the relationship, not just a single data point. An operation whose cost scales roughly linearly with round trips added tells you something different from one that falls off a cliff past a threshold, and you cannot tell which you are looking at from a single latency figure.

Producing these conditions honestly means using a real network-conditioning tool rather than guessing — something that actually delays and optionally constrains packets at the network layer, run between a real client machine and a real server or repository endpoint, rather than a script that merely waits before firing a request, which measures something else entirely. The specific tool matters less than the discipline of stating which one was used and what values it was configured with, so that someone else could set up the identical conditions and check the claim independently. A benchmark that says "we tested over a slow connection" without saying which tool, which values, and which direction the constraint was applied has not really told you what it did.

The operations that actually deserve separate timing

Enterprise Architect is not one operation performed repeatedly; it is a large surface of quite different interactions, and a benchmark that reduces all of it to a single "how fast is EA" number is throwing away the most useful part of the exercise. At minimum, four categories deserve to be measured and reported separately, because each one stresses a different part of the system and each one will respond differently to the difference between a local repository, a traditional remote repository, and a CelinQ-synchronised local repository.

Element creation is the first, and it matters because it is one of the more write-heavy, chatty operations an architect performs — a new element typically triggers not just an insert but a small sequence of related writes and reads as EA establishes the object, assigns identifiers, and updates whatever view is watching the package. Against a remote repository each of those steps can mean a round trip; against a local repository, whether that repository is a plain local file or a CelinQ-synchronised one, every one of those steps happens against storage sitting on the machine in front of you, and the round trip simply does not exist in the interactive path. Rename and property update is the second, worth measuring separately from creation because it exercises a narrower, more targeted write and is one of the most frequent single actions in a working session — the "quick fix" an architect makes without thinking about it, dozens of times a day, and therefore one of the operations where small per-action latency compounds fastest into a felt sense of sluggishness. Diagram opening is the third, and it deserves its own measurement because a diagram is not a single row; it is a collection of diagram objects, their geometry, their style overrides, and the underlying elements and connectors they represent, all of which typically have to be assembled before the canvas can render, making it one of the more request-heavy operations in ordinary use. Search and read operations are the fourth — a model search, a traceability query, simply browsing a tree and reading the properties of items as you go — and they matter because they are cognitively different from writes: an architect reading their way through a model to understand it before making a change is not tolerant of the same friction as someone who has already decided exactly what they are going to type.

A properly reported benchmark times each of these four categories independently, at each latency step, and reports them separately rather than folding them into one composite score. It is entirely plausible, and worth explicitly checking for, that some operations degrade sharply with added round-trip time while others barely move — a single average would smear that distinction away and leave you unable to reason about which of your own team's daily habits will actually be affected.

Initial clone and incremental synchronisation are not the same measurement

One methodological trap is specific to any local-first or synchronised architecture, CelinQ included, and it deserves its own section because getting it wrong produces genuinely misleading conclusions in either direction. The first time a local repository is established from a shared workspace — the initial clone — a real, sometimes substantial amount of data has to move, because there is no prior state to diverge from and the entire model has to be transferred once. Every subsequent working session, by contrast, only needs to move what actually changed: an incremental synchronisation carries a small delta, not the whole repository, because the local copy already holds everything else.

These are legitimately different operations with legitimately different cost profiles, and a benchmark — or a piece of marketing copy — that reports only one of them while implying it describes the other is misleading regardless of intent. Quoting a fast incremental sync time while glossing over what the initial clone costs makes the architecture look better than the full picture; quoting a slow initial clone time as though it represented ongoing daily performance makes it look worse than the full picture. Both numbers deserve to be published, clearly labelled, side by side, precisely because they answer different questions: the clone answers "what does it cost to onboard one more architect or one more machine," while the incremental figure answers "what does an ordinary working day feel like once you are already set up." An organisation evaluating a tool needs both answers and should not have to guess which one a headline number is describing. What changes when the repository lives beside the user depends on this distinction holding up honestly, not on the more flattering of the two numbers standing in for the whole story.

Median and P95, not an average that hides the worst moments

An average latency figure is one of the easiest ways to make a system look better than it feels to actually use, and it is worth being specific about why. Suppose ninety operations out of a hundred complete quickly and ten complete very slowly because of a retry, a contended resource, or an unlucky moment on a shared network. The average across all hundred can still look perfectly respectable, because the many fast operations mathematically absorb the few slow ones. But that average is not what the architect experiences. The architect experiences each operation individually, in sequence, across a working day, and the slow ones are the ones that get remembered, complained about, and eventually used to justify avoiding a piece of work altogether. A benchmark that only reports a mean is reporting the number least connected to how the tool actually felt to use.

This is why serious performance reporting in almost every other field that takes latency seriously — web services, databases, anything with a service-level expectation attached to it — reports percentiles rather than, or at minimum alongside, a mean. The median, the fiftieth percentile, tells you what a typical operation felt like. The ninety-fifth percentile tells you what the bad end of ordinary experience looks like — not a freak outlier, but the kind of slow moment that happens often enough that any regular user will run into it. Reporting both, across enough repeated trials to make the percentile meaningful — dozens of repetitions per condition at an absolute minimum, ideally more — is the difference between a number that describes an average day and a number that describes the days that actually make people complain. Any benchmark of Enterprise Architect over a network that reports a single mean and stops there has told you less than it appears to.

What reproducibility actually requires

A number that cannot be regenerated by someone else is, for evaluation purposes, indistinguishable from a number that was made up, even when it was measured honestly the first time. Reproducibility is not a courtesy extended to sceptics; it is what turns a one-off measurement into a fact other people can rely on. In practice it means a few concrete things. It means the test is scripted rather than performed by hand, so that the same sequence of operations runs identically each time rather than depending on how carefully a person clicked through a demo. It means any randomness in the test — which elements get created, in what order, against which part of a sample repository — is seeded, so that a stated seed value regenerates the exact same sequence of operations on a rerun, rather than a fresh random run producing a different, uncomparable result each time. It means the test exercises the real code path a working architect would actually take, not a simplified stand-in that happens to be faster or slower than the real thing for reasons unrelated to what is being measured. And it means the raw results are published in full, not just a summary — every trial, not a hand-picked run that happened to land favourably.

CelinQ already applies this discipline to a different measurement: the Fusion benchmark that exercises the merge engine, documented in the project's own engineering write-ups, runs one hundred thousand operations from five concurrent clients through the real storage layer and the real merge pipeline — nothing mocked or simplified out — with a fixed, published seed so the run is reproducible, and its invariants are asserted programmatically rather than eyeballed afterwards: the run fails outright if it detects any silent loss of work, any resurrection of a deleted element, or any client that fails to converge with the others. That benchmark reports what it actually found: zero silent loss, zero resurrection, full convergence across every client, and roughly seven in ten of the concurrent collisions it deliberately manufactured resolving automatically, with the remainder correctly escalated to a human decision rather than guessed at. It is a benchmark of correctness under concurrency, not of raw operation latency over a network, so it does not answer the question this article is about — but it is the right shape for what a network-latency benchmark should look like too: a fixed seed, the real code path, assertions instead of impressions, and the full method published next to the result rather than a headline number standing alone.

Why the local side of the comparison is not exempt from scrutiny either

It would be easy to read all of this as a checklist aimed only at a traditional remote-repository setup, but a fair benchmark applies exactly the same scrutiny to the local-first side of the comparison, CelinQ included. A local repository, whether plain or synchronised, is not automatically fast simply by virtue of being local; a poorly indexed local database, a machine with a slow disk, or a repository that has grown enormous without any housekeeping can all produce sluggish local performance that has nothing to do with the network at all, and a benchmark that only ever tests local performance on a fast, lightly-loaded machine with a modest repository is stacking the deck just as surely as a vendor who only ever tests their remote competitor over a punishing connection. The synchronisation layer itself has its own costs worth measuring honestly — the CPU and I/O overhead of the background agent watching for changes, the cost of applying an incoming changeset once it arrives, the brief moment of contention if a large deep reconciliation scan happens to run while someone is actively working. None of these costs are hidden by design, but none of them should be assumed away either. A methodologically honest benchmark of CelinQ's own architecture measures the apply cost of an incoming change on the receiving machine with the same rigour it applies to measuring a remote repository's round-trip cost, because both are real costs that a working architect will eventually feel, just in different places and for different reasons.

Reading a vendor's performance claim critically

Put the pieces together and a short, practical checklist falls out, useful the next time any vendor — a Pro Cloud Server reseller, a competing local-first tool, or CelinQ itself — hands you a slide with a performance number on it. Ask what hardware the test ran on, and whether it was disclosed in enough detail to matter, down to storage type. Ask what version of Enterprise Architect was used, since behaviour has shifted across releases. Ask how large the test repository was, in elements, packages and diagrams, and whether that size resembles anything like your own environment or was chosen because it is convenient to demo quickly. Ask exactly what network conditions were simulated, in milliseconds of round-trip time and any bandwidth constraint, and what tool produced those conditions. Ask which specific operations were timed, and be suspicious of a single blended number that does not distinguish creation from rename from diagram-opening from search. Ask whether the reported figure is a mean, a median, or a percentile, and ask to see the ninety-fifth percentile specifically, since that is where the daily frustration actually lives. Ask whether initial clone cost and incremental synchronisation cost were kept separate or quietly merged into one figure that flatters whichever number is smaller. And ask, plainly, whether the test is reproducible — whether you, or an independent third party, could rerun it and expect to land somewhere close to the same result.

None of these questions require deep technical expertise to ask. They require only the willingness to treat a performance claim the way you would treat any other claim made by someone with a reason to want you to believe it — as worth checking, not worth dismissing, but worth checking. A vendor confident in their own numbers will usually be glad to answer every one of these questions in detail, because the detail is what makes the number credible in the first place. A vendor who deflects the questions, or who has genuinely never measured the thing this precisely, has told you something too, just not in the form of a number.

Where CelinQ stands today, honestly

The honest position, stated plainly rather than buried in a footnote, is that CelinQ has not yet published the specific controlled benchmark this article describes — the one that times element creation, rename, diagram opening and search, separately, across a local repository, a traditional remote repository and a CelinQ-synchronised repository, at fixed 20/50/100/200 millisecond latency steps, on disclosed hardware, against a disclosed Enterprise Architect version and repository size, reporting median and ninety-fifth-percentile figures with clone and incremental synchronisation kept apart, and with the raw results and the seed available for anyone to rerun. What does exist today, documented and reproducible in exactly this spirit, is the Fusion correctness benchmark described above, along with the architectural facts that make the latency argument plausible in the first place: that ordinary reads and writes against a CelinQ-managed local repository happen against local storage with no round trip in the interactive path, and that synchronisation happens asynchronously in the background on a cadence — a short debounce after an edit, a periodic safety pass, an occasional deeper reconciliation — designed to keep the shared workspace current without ever sitting in front of the person trying to work. Those are architectural claims, reasonable ones, and they are why this benchmark is worth running. They are not, by themselves, the measured latency numbers a rigorous comparison would produce, and this article has deliberately not dressed them up as such.

That gap is worth naming rather than smoothing over, because the entire argument of this piece falls apart the moment it starts making an exception for itself. The right response to an unpublished number is not to publish a worse one out of impatience; it is to build the benchmark properly, publish the method alongside the result, and invite exactly the scrutiny described above. Until that benchmark exists in that form, the responsible thing to tell a prospective customer, or a curious architect reading this series, is precisely what this article has told you: here is what a trustworthy version of that measurement would require, here is why each piece of it matters, and here is the standard we intend to be held to when we publish it. Anything less specific than that would be exactly the kind of unverifiable performance sentence this whole article has been arguing against, and a series that spends this much effort discussing what genuinely depends on the network and what does not loses all credibility the moment it starts making unverifiable claims of its own.

If you are evaluating Enterprise Architect performance over a network — whether you are looking at Pro Cloud Server, at a VPN-tunnelled remote database, or at a local-first product like CelinQ — do not accept a summary sentence from anyone, including us. Ask for the method. A vendor who has actually done the work will hand it to you without hesitation, because the method is what makes the number worth believing, and a number without one is not a benchmark. It is a hope, printed in a font that looks like data.