Testing methodology
Every benchmark, metric, and broker evaluation on NoDealingDesk follows a standardized testing protocol designed for statistical rigor, reproducibility, and transparency. This page documents our complete methodology.
Core principles
Controlled test environment
All tests run from VPS co-located in LD4 (London) and NY4 (New York) using identical hardware configurations. Network conditions are monitored and logged alongside test data to account for external factors.
Standardized order parameters
Test orders use 0.01 lot market orders on EURUSD, GBPUSD, and USDJPY. Orders are placed during London/NY session overlap (13:00-17:00 UTC) at fixed 30-second intervals to ensure statistical comparability.
Nanosecond-precision timestamps
All timing measurements use NTP-synchronized clocks with nanosecond precision. Latency is measured as the full round-trip from MT5 API order submission to server fill confirmation.
Statistical rigor
Minimum sample size of 1,000 orders per broker per metric. Results include confidence intervals, standard deviations, and percentile distributions. Outliers are identified but never silently removed.
Broker anonymization
Brokers are identified by letter codes (Broker A, B, C) in published reports. Named broker profiles on the site use publicly available execution data and broker-disclosed specifications only.
Reproducibility
Every report includes sufficient detail for independent verification. Test parameters, time windows, VPS specs, and statistical methods are fully documented. We welcome third-party replication.
Testing pipeline
Account setup
Live accounts opened with standard retail conditions. No special arrangements or institutional pricing. Demo accounts are never used.
VPS configuration
Dedicated VPS instances provisioned in LD4 and NY4. NTP time synchronization verified. Network baseline latency measured to broker servers.
Data collection
Automated test scripts place orders via MT5 API. Every order logs: submission timestamp, fill timestamp, requested price, fill price, order type, symbol, and server response code.
Quality control
Raw data validated for completeness, timestamp integrity, and network anomalies. Orders placed during known connectivity issues are flagged but retained with annotations.
Statistical analysis
Metrics computed per our defined formulas (see individual metric pages). Confidence intervals calculated at 95%. Distribution normality tested with Shapiro-Wilk.
Report publication
Results compiled into standardized report format. Internal review for accuracy. Published with methodology reference, limitations section, and changelog history.
Known limitations
- Micro lots only -- Tests use 0.01 lots. Execution quality may differ for standard (1.0) or institutional lot sizes.
- Major pairs only -- Current tests cover EUR/USD, GBP/USD, USD/JPY. Cross pairs, exotics, and commodities are not yet included.
- Market orders only -- Limit and stop order execution quality is not measured in current methodology.
- Retail conditions -- Results reflect standard retail account execution, not institutional or prime brokerage conditions.