Microsoft UI Automation APIs, also known as the MSUIA or UIA libraries were released as a part of .NET Framework 3.0 and above. These APIs are successors of the MSAA (Microsoft Active Accessibility ) classes.
The MSUIA libraries expose control patterns for each control, actions and properties for these controls. Using these control pattern classes, MSUIA lets the users assess and manipulate values of controls on your system.
The Search in UIA is in a tree structure where Root Element (Desktop) is the parent of all the elements. All the windows on your desktop are Children of Root Element which will have their own children elements like button, link etc. All the children and children of children's ( I'm trying to say that all the elements in all the windows open on your system at a time) are Descendants of root element
There are many tools available to find the elements and their properties, but ideally we use the Inspect.exe. Inspect is present in the bin folder of Windows Kits (if installed) in your program files.
When you open Inspect.exe, you will see a downward tree structure with "Desktop" pane as the root and all the windows as chidren within it. When you expand a window element, you will be able to view the elements within that window.
This is how inspect window looks like:
Let's get started with using MSUIA libraries
- First thing , I would suggest start a new project in Visual Studio (ideally UnitTest Project) and try out all the classes I would be walking you through in the rest of this article.
- Secondly, download MSUIA references and add them to your project . The DLL are as following :
1. UIAutomationProvider.dll and UIAutomationTypes.dll [ Provider API]
2. UIAutomationClient.dll and UIAutomationTypes.dll [Client API]
3. UIAutomationCore.dll
4. UIAutomationClientSideProviders.dll
- Thirdly in a new class add the namespace"using System.Windows.Automation;"
Now you are ready to go.
The following few classes are important for you to know about when planning to work on these libraries.
1) AutomationElement : This is a sealed class, which saves the properties of a control in its object.
RootElement is a keyword which returns the desktop element, under which all the elements/controls are searched for.
2) AutomationElementCollection contains objects of multiple AutomationElement. If you want to find all controltype= button inside a parent element A, you can use this class object while AutomationElement always returns the first objects it finds with the properties mentioned.
3) PropertyCondition saves conditions on the basis of which an object is identified.
For your better understanding, the signature of one of the methods present in the PropertyCondition class looks like.
// takes a property name like 'Class name' , 'id', 'name' etc and its value .
4) ControlPatterns : Every control will support a/few control pattern(s). If a control supports a pattern or not, can be seen on the right hand column of Inspect.exe when you select the control. The is<patternname>Available property will be set to true. For example, for a button, the isInvokePatternAvailable property is set to true.
Every control pattern has a class in MSUIA libraries example :
GridPattern.cs
ExpandCollapsePattern,cs
InvokePattern.cs
TablePattern.cs
ValuePattern.cs
If you want to click on a button (which has invoke pattern available), perform the following steps:
Similary, if you want to enter a text in a textBox (which has value pattern available), you use the ValuePattern class object which has a method called SetValue.
To get the current properties involved with any control, you can use the property 'Current' which is a property in each Pattern class.
See the following code .
Start with taking a windows/WinForms based project and try performing the above actions on their controls.
Hope this tutorial works. Write back to me or comment for more detailed information/tutorial.
Happy learning :)
