External automation

[martonmiklos]

At some point I would like to use the AgIsoVirtualTerminal for some automated implement testing.

What do you think about adding an optional (compile time selectable) telnet like inteface which would be capable to do the following:

• Taking screenshot to a specific file
• Pressing softkeys by index
• Entering input numbers or texts

I know these automation could be done with screen image processing and UI automation tools, but it would be more "portable" if these features could be boundled into the terminal itself. For e.g. if we change how the number inputs look like then these image screen processing based automations would likely need be tailored.

[gunicsba]

This could allow Functional Testing using the final product and testing it against real equipment(s).

For example today I realized that the softkey switcher doesn't always trigger a repaint of the softkeys. (And sometimes neither does the changeWorkingSet.)

[martonmiklos]

@GwnDaan @ad3154 any thoughts on this one?

[ad3154]

I think that a scriptable automation interface probably makes sense - I have heard that request from many people, and such an interface would allow us to better test our own client stuff as well. I don't think a telnet-like interface by itself makes much sense personally, but something scriptable certainly does. Interactive is a lot harder than scripting, and less flexible to automate I think. I think we would need to really think through whatever we do to make it as generic as possible, able to be used for 100% of all defined commands with entirely arbitrary lists of steps to execute, able to run in a github action, have quantifiable image-based and CAN message based pass criteria, etc

[martonmiklos]

I think that a scriptable automation interface probably makes sense - I have heard that request from many people, and such an interface would allow us to better test our own client stuff as well. I don't think a telnet-like interface by itself makes much sense personally, but something scriptable certainly does. Interactive is a lot harder than scripting, and less flexible to automate I think. I think we would need to really think through whatever we do to make it as generic as possible, able to be used for 100% of all defined commands with entirely arbitrary lists of steps to execute, able to run in a github action, have quantifiable image-based and CAN message based pass criteria, etc

What do you mean exactly under scriptable? Have some sort of script interpreter compiled into the app, or integration with common script languages (i.e. python)?

[ad3154]

What do you mean exactly under scriptable? Have some sort of script interpreter compiled into the app, or integration with common script languages (i.e. python)?

Probably the former - we need some way to tell the application to do things and expect certain behaviors, then return if they pass or fail.

For example, let's say we're writing a test where we want to interrogate the behavior following test sequence:

1. The object pool gets uploaded and activated and has no errors
2. A container in the pool which is not hidden is shown
3. The VT responds correctly to a command to hide the container
4. The container is hidden

In a hypothetical script, I'd want to be able to do something like

UPLOAD <file path> # or if a real implement is doing it, maybe WAIT 300 then ASSERT CONNECTED TRUE
ASSERT <object ID> ATTRIBUTE EQUAL <attribute number> <attribute value>
VT_COMMAND <command number> <value>
ASSERT <object ID> ATTRIBUTE EQUAL <attribute number> <attribute value>

This is just example syntax but hopefully you get what I mean. Something like this would make for an easy and very flexible interpreted script on the VT side, and the VT could handle all the pass/fail criteria itself since it knows the entire state of the object pool, the active working set's cached bitmap being displayed, and knows all the responses it would send to the client.

At least, this is what I had in mind originally.