Testing & Go-Live: Final Validation and Deployment

Part 5 of 5: Optix Migration Planning Series

Link to Part 4: Development Best Practices

The difference between successful migrations and problematic ones comes down to testing rigor and deployment planning. Engineers who skip comprehensive testing end up debugging at 2 AM during production runs. Engineers who test thoroughly commission smoothly.

This post covers Factory Acceptance Testing, Site Acceptance Testing, operator training, go-live planning, and post-launch support. The practices that separate smooth deployments from emergency troubleshooting sessions.

Testing Philosophy

Test to Find Problems, Not Prove Success

Wrong mindset: "Let's show the customer it works."

Right mindset: "Let's break this before the customer does."

The goal is finding and fixing issues before production. Every bug found in testing is a bug not found during commissioning. Testing isn't overhead, it's risk reduction.

Test with intent to break things. You'll build confidence when you can't find errors.

Developer Testing

Continuous testing during development catches obvious issues early.

What you're testing:

• Individual screens load and display correctly
• Tag bindings read/write as expected
• Navigation works (forward, back, popups)
• Buttons trigger correct actions
• Emulator runs without crashes

Frequency: Every time you build something new. Test immediately, not later.

Why it matters: Catching issues in development is 10x faster than catching them during FAT. Fix while context is fresh.

Factory Acceptance Testing (FAT)

Comprehensive functional testing at your OEM facility before machine ships to customer. This is where you validate the complete application systematically.

Environment: Machine at OEM facility, real or simulated hardware, PLC connectivity, all integrations present.

Who attends: Engineering team, test lead, project manager. Internal technicians who will support customer startups. Customer operators if possible (best source of usability feedback).

FAT is your last chance to find issues in a controlled environment. Invest the time.

Site Acceptance Testing (SAT)

Testing at customer's facility during commissioning with variables you couldn't simulate at your OEM facility.

Environment: Customer's facility, their production network, their actual equipment, their operators.

Why separate from FAT: Customer's environment introduces variables you can't fully replicate at OEM facility. Their network security, their actual equipment integration, their operator workflows, their environmental factors (lighting, glare, noise).

Issues found at SAT that weren't in FAT: Customer's firewall blocking OPC UA port. Screen unreadable due to glare in their facility. Touch targets too small for their operators with gloves. Workflow doesn't match how their operators actually work.

SAT validates that FAT success translates to customer's production reality.

Template Testing Insight

When testing template-driven applications, failure patterns tell you where the problem lives.

Issue on one instance: Alias configuration or instance-specific problem. Check that instance's parameter values.

Issue on multiple instances: Template-level problem. Fix the Type, all instances inherit the correction.

This distinction helps you debug efficiently. Don't fix the same issue 50 times on 50 instances when the root cause is in the template.

Factory Acceptance Testing

FAT validates complete functionality before hardware leaves your facility. Think of FAT as comprehensive system validation, not just "does it turn on."

Functional Validation

Core application functionality works as designed.

Test coverage:

• Every screen accessible through navigation
• All buttons and controls trigger correct actions
• Tag bindings read values from PLC
• Tag bindings write values to PLC
• Alarms trigger on correct conditions
• Alarms acknowledge properly
• Trends display historical data
• Data logging captures information

Focus: Does the application do what it's supposed to do?

Integration Testing

Systems beyond the HMI work together correctly.

Test coverage:

• PLC communication stable over extended periods
• Historian data logs correctly
• Database queries execute successfully
• External APIs respond (if applicable)
• Recipe management saves/loads correctly
• Report generation works

Focus: Does the application integrate with everything it needs to integrate with?

Performance Validation

Application performs acceptably under realistic conditions.

Test coverage:

• Screen load times under 2 seconds (target)
• Tag update rates acceptable for process
• Application stable over 8+ hour run

Focus: Will this perform acceptably in production?

Note: Emulator performance differs from panel deployment. Always validate on target hardware or representative hardware.

User Acceptance

If operators can attend FAT, get their feedback.

Test coverage:

• Operators can find needed information
• Operators can perform required tasks
• Workflows match how operators actually work
• Interface is intuitive (or familiar if hybrid approach)
• Training materials are accurate

Focus: Will operators be successful using this system?

Why operators matter: Engineers think logically. Operators think practically. Catch usability issues before deployment. Operator feedback during FAT is worth 10x its weight compared to feedback during commissioning.

Site Acceptance Testing

SAT happens at customer's facility during commissioning and introduces environment variables that couldn't be fully tested at your OEM facility. This is validation in the customer's actual deployment context.

Production Environment Variables

Customer's facility differs from OEM factory testing in measurable ways.

What's different:

• Customer's network (firewalls, VLANs, security policies)
• Their equipment integration (actual I/O, upstream/downstream process connections)
• PLC program in their environment
• Customer's facility lighting and environmental conditions
• Their operator PPE and working requirements (gloves, safety glasses, facility-specific gear)

Test coverage:

• Network communication stable in customer's infrastructure
• Firewall rules allow required traffic
• Screen readable in customer's facility lighting
• Touch targets accessible with their operators' PPE
• Hardware performance adequate in their environment

Common issues found: Customer's network security blocking ports. Screen glare from their facility windows. Touch targets too small for their gloves. PLC tag addresses differ from development system.

Operator Validation

Customer's operators using their actual workflows validate practical usability.

Test coverage:

• Customer operators comfortable with navigation
• Common tasks are intuitive (or familiar if hybrid approach)
• Workflows match their actual practice
• No confusion about functionality
• Operators feel confident using system

Focus: Does this work for the people who will actually use it at the customer's facility?

Final Integration

Equipment integration validated with customer's actual hardware and production systems.

Test coverage:

• All I/O points read correctly from customer's equipment
• Commands execute on customer's equipment as expected
• Interlock logic functions correctly in their environment
• Safety systems work as designed
• Emergency stop procedures validated with actual equipment

Focus: Does the HMI control the customer's actual equipment correctly in their production environment?

Operator Training

Training scope depends heavily on which migration strategy you used from Part 3.

Two training audiences: Your internal technicians (who support machine startups) and customer operators (who run the machines in production). This section focuses primarily on customer operator training, though internal technicians benefit from similar coverage during FAT.

Strategy impact: Hybrid's minimal training burden is a key value proposition for OEMs with trained customer base. Full redesign requires accepting significant training investment.

Hybrid approach

Training focuses on what changed, not comprehensive re-training.

Cover: New navigation elements (if changed). New features added (multi-language, web access). Where familiar things moved (if anything relocated). Improved workflows (if redesigned).

Full redesign approach:

Comprehensive training required. Operators learning entirely new interface.

Cover: Complete navigation structure. All screens and functions. New workflows. New features. Troubleshooting basics.

Strategy matters: Hybrid's minimal training burden is a key value proposition for OEMs with trained customer base. Full redesign requires accepting training investment.

Training Content

Regardless of strategy, certain topics need coverage.

Core training:

• How to navigate between screens
• How to find equipment and information
• Acknowledging alarms and responding to issues
• Changing setpoints and parameters
• Viewing trends and historical data
• Loading recipes (if applicable)

Troubleshooting basics:

• What to do if screen freezes
• How to restart application
• Who to contact for issues

New features (if applicable):

• Multi-language switching
• Web/mobile client access
• Advanced features not in ME

Training method: Hands-on practice beats classroom lecture. Let operators use the system, practice common tasks, ask questions, build muscle memory. Supplement with documentation, don't replace hands-on time.

Documentation Deliverables

Training sessions are temporary. Documentation is permanent reference.

For operators and customers:

• Operator manual (comprehensive reference)
• Quick reference guide (common tasks, troubleshooting)
• Screen-by-screen navigation guide (if helpful)

For internal use and validation:

• Technical specifications (system architecture, tag lists, alarm configurations)
• Commissioning procedures (startup, validation, acceptance criteria)
• Maintenance and troubleshooting guide (for engineering support)

Documentation quality impacts long-term support burden. Invest in clear, accurate documentation upfront. It pays dividends every time someone needs to reference it.

Go-Live Planning

For OEMs, "go-live" means shipping machines with Optix panels to customers. This is different from end-user cutover. You're transitioning your standard panel offering while managing existing ME commitments.

Deciding Which Machines Get Optix

Not every machine needs to migrate simultaneously. Strategic rollout reduces risk.

Ship with Optix:

• New machine types (no legacy expectations)
• New customers (no ME installed base to match)
• New product lines (fresh start, no consistency burden)
• High-volume machines (template ROI justifies investment)

Consider staying with ME:

• Sister machines for existing customers (Customer A has 10 ME machines, buying #11 - match their installed base)
• Low-volume legacy machines (migration cost doesn't justify frequency)
• Customers with firm ME requirements
• Rush orders where Optix risk outweighs timeline

Reality: Expect a period where you're shipping both ME and Optix simultaneously. Different customers, different machine types, different situations. This is normal.

First Shipment Risk Management

The first customer to receive Optix machine carries highest risk. Choose carefully and prepare thoroughly.

Choosing first customer:

• Friendlier relationship (not your most demanding customer)
• Reasonable timeline (not rush commissioning)
• OEM engineer can be on-site for startup
• Customer willing to provide feedback
• Lower-stakes application (not their most critical line)

Risk mitigation:

• Have ME panels as backup (first 2-3 shipments)
• OEM support scheduled for commissioning
• Clear escalation path if issues arise
• Don't ship to highest-profile customer first
• Test machine thoroughly at OEM facility before shipping

Preparation checklist:

• FAT complete at OEM facility, all issues resolved
• Internal technicians trained on Optix
• Customer documentation finalized (operator manuals, quick reference)
• Technical specs complete for commissioning support
• Support team scheduled and available

Supporting Customer Commissioning

Customer commissioning is your real "go-live" moment. This is where Optix faces production reality.

OEM support during commissioning:

• Engineer on-site (or immediately available remote) for first 2-3 customer installations
• Direct line to engineering team if issues arise
• Rapid response to customer questions
• Daily check-ins during commissioning week
• Issue tracking for lessons learned

What to monitor:

• Integration with their upstream/downstream equipment
• Network connectivity in customer's environment
• Operator acceptance and comfort level
• Any issues that weren't found during SAT

Customer production startup: Once customer starts production with Optix machine, monitor closely for first week. Rapid response to issues builds confidence. Track everything for lessons learned.

Transition to normal support: After first week of production, support transitions to standard warranty/service model. Most issues surface in first days of production use.

Lessons Learned

Learning happens through real customer deployments, not just factory testing. Every customer installation teaches something valuable.

Capturing Feedback

The first few customer installations reveal issues that testing couldn't predict.

Sources of learning:

• Operator feedback during commissioning (workflows that don't match their practice)
• Issues during first week of customer production (real usage patterns emerge)
• Customer-specific requirements (features they need that weren't anticipated)
• Environmental factors unique to their facility (lighting, network, integration challenges)
• Service technician feedback (what's difficult to troubleshoot or support)

Track everything: Description of issue, when it occurred, customer environment, resolution, root cause. Build institutional knowledge base.

Iterating Templates

Customer feedback drives template improvements that benefit all subsequent machines.

Common evolution patterns:

• Customer requests become reusable optional features (add to configuration object, enable per machine)
• Edge cases get built into core templates (don't rebuild for each customer, solve once in template)
• Workflow improvements from operator feedback (navigation tweaks, screen layout adjustments)
• Integration patterns from diverse customer environments (network configurations, upstream/downstream systems)

Internal learning matters too: Your engineering team is also learning Optix during first few machines. Capture best practices, document gotchas, share knowledge. By tenth machine, team efficiency is dramatically higher.

Building Institutional Knowledge

Systematic documentation of lessons prevents re-learning same issues.

What to document:

• Customer-specific requirements and how they were addressed
• Common commissioning issues and their solutions
• Features that customers value most (prioritize these in future templates)
• Features that customers don't use (consider simplifying or removing)
• Best practices for supporting customer commissioning

Debrief after each installation: What went well? What surprised us? What would we change? What should we add to templates?

Conclusion

Investing in testing leads to smooth customer commissioning, confident operators, and successful migration. The quality of your testing determines the perception of the migration.

Each customer installation should be smoother than the last. After a dozen deployments you'll wonder why you didn't migrate sooner.

Critical practices:

• Test to find problems, not prove success
• Factory Acceptance Testing at OEM facility before shipping
• Site Acceptance Testing at customer facility during commissioning
• Training appropriate to migration strategy (customer operators focus)
• Strategic rollout (choose first customers carefully)
• Support during customer commissioning and early production
• Capture lessons learned from every customer deployment

Series Complete

You now have a framework for ME to Optix migration:

Part 1: When to Migrate
Part 2: Assessment Phase
Part 3: Migration Strategies
Part 4: Development Best Practices
Part 5: Testing & Go-Live

The architecture that underpins everything: The Architecture Advantage: Why Optix Template Design Determines ROI

Next step: Apply this knowledge to your migration. First machine is hardest. Second machine is easier. By fifth machine, you're wondering why you didn't migrate sooner.

Template-driven development delivers ROI. But only if you design architecture properly, test thoroughly, and deploy systematically.

Need Deployment Support?

ASQI offers implementation support for Optix migrations:

• FAT and SAT protocol development
• On-site commissioning assistance
• Operator training program design
• Post-launch troubleshooting and optimization

Contact ASQI →

Related Articles

Migration Execution Guide — The project-planning checklist that precedes testing.

Optix Tips and Tricks — Common runtime pitfalls to check during your testing phase.

The Architecture Advantage — If testing reveals architecture issues, revisit template design fundamentals.