AutomationTestToolset.AutomationTestToolset

Automation test discovery and execution toolset.

Wraps the IAutomationControllerManager API (the same backend the Session Frontend uses) to let MCP clients list available tests, run them, and retrieve results.

Typical workflow: 1. DiscoverTests() once per session to initialize workers and load the test list. 2. ListTests() to find tests by name or tag. 3. RunTests() with the desired test names. 4. GetTestStatus() / GetTestResults() to monitor and retrieve results. 5. StopTests() to abort if needed.

7 tool(s).

AutomationTestToolset.AutomationTestToolset.DiscoverTests

Initialize automation worker discovery and load the test list. Must be called once before ListTests or RunTests. Takes several seconds as it discovers the local automation worker and enumerates all registered tests. Returns an async result that completes with a JSON status object when tests are available, or an error if discovery fails.

Input schema

{
  "properties": {
    "bForceRediscover": {
      "default": false,
      "description": "When true, bypass the cached report tree and re-poll workers. Used after reloading Python test modules mid-session.",
      "type": "boolean"
    }
  },
  "type": "object"
}

Output schema

{
  "properties": {
    "returnValue": {
      "description": "Value of the result.",
      "type": "string"
    }
  },
  "required": [
    "returnValue"
  ],
  "type": "object"
}

AutomationTestToolset.AutomationTestToolset.GetTestResults

Get detailed results for the current or most recent test run. Requires DiscoverTests() to have completed. Returns a JSON object with per-test state, duration, errors, and warnings.

Input schema

{
  "type": "object"
}

Output schema

{
  "properties": {
    "returnValue": {
      "type": "string"
    }
  },
  "required": [
    "returnValue"
  ],
  "type": "object"
}

AutomationTestToolset.AutomationTestToolset.GetTestStatus

Get a lightweight status snapshot of the automation controller. Requires DiscoverTests() to have completed. Returns a JSON object with the controller state, enabled test count, and completion/pass/fail counts.

Input schema

{
  "type": "object"
}

Output schema

{
  "properties": {
    "returnValue": {
      "type": "string"
    }
  },
  "required": [
    "returnValue"
  ],
  "type": "object"
}

AutomationTestToolset.AutomationTestToolset.ListTests

List available automation tests. Requires DiscoverTests() to have completed. Returns a JSON object: {"tests": ["path1", ...], "total": N, "returned": N}.

Input schema

{
  "properties": {
    "limit": {
      "default": 200,
      "description": "Maximum number of tests to return (0 = unlimited, default 200).",
      "type": "integer"
    },
    "nameFilter": {
      "description": "Optional substring filter applied to the test's full path.",
      "type": "string"
    },
    "tagFilter": {
      "description": "Optional substring filter applied to the test's tags.",
      "type": "string"
    }
  },
  "required": [
    "nameFilter",
    "tagFilter"
  ],
  "type": "object"
}

Output schema

{
  "properties": {
    "returnValue": {
      "type": "string"
    }
  },
  "required": [
    "returnValue"
  ],
  "type": "object"
}

AutomationTestToolset.AutomationTestToolset.RunTests

Run a set of automation tests by name. Requires DiscoverTests() to have completed. Starts executing the specified tests and returns an async result that completes with a JSON summary when all tests finish.

Input schema

{
  "properties": {
    "testNames": {
      "description": "Array of full test paths as returned by ListTests.",
      "items": {
        "type": "string"
      },
      "type": "array"
    }
  },
  "required": [
    "testNames"
  ],
  "type": "object"
}

Output schema

{
  "properties": {
    "returnValue": {
      "description": "Value of the result.",
      "type": "string"
    }
  },
  "required": [
    "returnValue"
  ],
  "type": "object"
}

AutomationTestToolset.AutomationTestToolset.RunTestsByFilter

Run automation tests selected by a filter expression. Requires DiscoverTests() to have completed. Much faster than RunTests when targeting a large batch because the engine narrows the report tree in a single pass instead of running a per-leaf membership check against the requested name list.

Filter syntax (multiple expressions joined by '+'): "StartsWith:System.Engine" prefix match against the full test path "^Foo" prefix anchor (equivalent to StartsWith:) "Bar$" suffix anchor "Substring" bare token matches anywhere in the path "Group:Smoke" expand a named group from AutomationControllerSettings ini Groups

Returns an async result that completes with the same JSON summary as RunTests.

Input schema

{
  "properties": {
    "filterExpression": {
      "description": "Filter expression as described above.",
      "type": "string"
    }
  },
  "required": [
    "filterExpression"
  ],
  "type": "object"
}

Output schema

{
  "properties": {
    "returnValue": {
      "description": "Value of the result.",
      "type": "string"
    }
  },
  "required": [
    "returnValue"
  ],
  "type": "object"
}

AutomationTestToolset.AutomationTestToolset.StopTests

Stop all currently running tests. Requires DiscoverTests() to have completed. If a RunTests async result is pending, it will be completed with an error.

Input schema

{
  "type": "object"
}

Output schema

{
  "properties": {
    "returnValue": {
      "description": "True if the stop request was issued successfully.",
      "type": "boolean"
    }
  },
  "required": [
    "returnValue"
  ],
  "type": "object"
}