product: maestro audience: test-developer authority: normative

What is Accordion Maestro?

Accordion Maestro is a modern test orchestration platform for production hardware testing — a replacement for NI TestStand. It coordinates test execution across physical instruments and devices, records structured measurement data, provides a web-based operator UI, and integrates with MES systems for factory traceability.

Maestro does not replace your test code. It replaces the glue around it — the sequencer, the operator interface, the result storage, and the station configuration layer. Your .NET or Python test functions stay exactly as they are; Maestro calls them via gRPC and records their results.

How Maestro compares to other tools

Capability	Accordion Maestro	NI TestStand	pytest / MSTest / xUnit	Robot Framework
Primary purpose	Production hardware test orchestration	Production hardware test orchestration	Software unit / integration testing	Software acceptance / RPA testing
Test definitions	YAML (version-controlled in Git)	Proprietary `.seq` files (binary)	Code (Python / C#)	Keyword-driven `.robot` files
Test code language	.NET and Python (plain functions)	LabVIEW, C, .NET, Python	Native to framework	Python / Java / .NET keywords
Measurement limits	Built-in: numeric ranges, boolean, string, log	Built-in (NI limits model)	Manual `Assert` calls — no standard limits model	No built-in measurement concept
Step-level verdict	Automatic per-step PASS / FAIL / SKIPPED / ABORTED	Automatic per-step	Per-test pass/fail only	Per-keyword pass/fail
Retry / repeat / abort	Declarative YAML fields (`retry:`, `repeat:`, `run_on_abort:`)	Sequence editor flow control	Manual code loops or plugins	Retry via libraries
Operator UI	Web-based (Blazor) — any device, any OS	Windows desktop application	Terminal / IDE	Terminal / browser (RF reports)
Operator prompts	Built-in `prompt` step with images, buttons, value input	Dialogue steps in sequence	Not applicable	Not applicable
Station configuration	Database-backed (`cfg.*` per station)	Station globals / config files	Environment variables or fixtures	Variable files
Result storage	PostgreSQL — every measurement as a queryable row	Database or ATML/XML export	JUnit XML or terminal output	XML / HTML reports
MES integration	Built-in routing, result reporting, serial gating	Built-in (NI InSight, custom)	Not applicable	Not applicable
Traceability	Git commit hash + package version on every execution	File-based traceability	Not designed for production traceability	Not designed for production traceability
Licensing	No per-seat runtime fees	Per-seat runtime license	Open source / free	Open source / free
Platform	Docker — Linux or Windows	Windows only	Cross-platform	Cross-platform
Real-time monitoring	SignalR live step progress	TestStand operator interface	Not applicable	Not applicable

When to use what

Use Maestro when you are building a production test system that involves physical hardware, instruments, measurement limits, operator interaction, station-specific configuration, and/or MES traceability.
Use pytest / MSTest / xUnit for software unit tests, integration tests, and CI pipelines. You can (and should) use MSTest alongside Maestro to debug your .NET test functions locally before deploying — see sdk/mstest-debugging.md.
Use Robot Framework for software acceptance testing, browser automation, or RPA workflows. It is keyword-driven and designed for a different problem domain.

Concept mapping from other ecosystems

Your background	You are used to	Maestro equivalent
NI TestStand	Sequence file → steps → substeps	YAML `test:` → `steps:` → `type: sequence`
	Step types (Action, NumericLimit, Pass/Fail)	`runner: dotnet`/`python` + `measurement:` block
	Station globals	`cfg.*` station configuration variables
	`RunState.Root.Parameters`	`variables:` block + `{{var}}` templates
	Sequence editor GUI	VS Code + YAML schema autocomplete
	Operator Interface	Test Monitor web page
	TestStand Result Collection	PostgreSQL measurement rows
pytest	`test_*.py` functions	Python functions in `python_modules/` — no `test_` prefix needed
	`conftest.py` fixtures	YAML `setup:` / `teardown:` steps
	`@pytest.mark.parametrize`	YAML `repeat:` with `{{__iteration__}}`
	`assert value > limit`	YAML `measurement:` with `low_limit:` / `high_limit:`
	`pytest.ini` / `pyproject.toml`	`package.json`
MSTest / xUnit	`[TestMethod]` / `[Fact]`	One YAML step per method
	`[ClassInitialize]` / constructor	YAML `setup:` steps
	`[ClassCleanup]` / `Dispose`	YAML `teardown:` steps with `run_on_abort: true`
	`Assert.IsTrue(v >= low && v <= high)`	YAML `measurement:` with limits — auto-evaluated
	`.runsettings`	Station config (`cfg.`) + execution metadata (`exec.`)

Step verdicts

Verdict	Meaning
`PASS`	Step succeeded; all measurements within limits
`FAIL`	Measurement out of limits or runner returned failure
`SKIPPED`	Not executed (disabled, precondition false, or `force: skip`)
`ABORTED`	Could not complete (timeout, exception, or precondition error)

Test verdict

Verdict	Condition
`PASS`	Every executed step is PASS or SKIPPED
`FAIL`	One or more steps FAIL, none ABORTED
`ABORTED`	One or more steps ABORTED (overrides FAIL)