ABB Operator Graphics Conversion to PI Vision

The ABB Operator graphics is used to view the real time of industrial process data. It utilizes the COM infrastructure in the form of control (*.ocx) that can be embedded into the Windows OS graphics viewing infrastructure.

When the user decides to modify the screen, it uses the Visual Basic Framework to perform editing the task and then when it is deployed, the source is compiled into *.ocx. You can examine this mechanism in my earlier article about ABB graphics deployement.

Since the original graphics is always compiled from the source, it is possible to retrieve the *.ctl and *.ctx file to be parsed by the conversion program to generate the PI Vision Display Script.

There are certain objects that is dynamically generated based on given parameter, for example the AdvantCone object which is part of AdvBpoPrimitives.dll control as well as AdvBpoExtendedPrimitives.dll control for a more complex objects.

For the above objects, the required property can be obtained by parsing the binary file of the related *.ctx file, for example the ConeStrength property can be obtained from certain offset inside *.ctx file.

The conversion program then can create the object on the fly using the COM interop methods and assigned the properties that is already accessed earlier from *.ctx binary file.

To actually generate the image, the conversion program then can use the Draw method of IViewObject which is part of OLE infrastructure, and can be saved into the bitmap file.

This bitmap file then can be transformed into *.PNG file and using base 64 encoding to generate the base64 text string that is handy for the PI Vision script’s image file.

Analysis ABB Graphics Editor Deployment Mechanism

Each of the ABB Operator screen can be created using its build in Graphics Editor by clicking the edit menu as follows:

It will then activates the customized version of Visual Basic 6 and the displayed graphics can then be deployed into the ABB Plant Explorer’s workspace:

When the Edit menu is processed, the generated Visual Basic’s project and other supporting files is located in the C:\OperateITTemp1\ActiveGraphics. To retrieve these files, the ABB’s Plant Explorer application (AfwWorkplaceApplication.exe) will perform COM remote interface invoke call to AfwFsdNode.exe, which in turn will perform communication with the File Set Distribution Server (AfwFsdSrv.exe) to get the required Visual Basic files required for Graphics Editor.

The files itself is located in the server’s C:\OperateITData\FsdServer. Now, one of interesting feature is to create a customized Visual Basic project that has the debug symbol for generated ABB graphics object. This is done by perform setting of CodeViewDebugInfo so that Visual Basic can generate the debug symbol file (*.PDB) for the graphics object.

You can see that from the above picture there’s no menu to perform setting for debugging purposes, but you can perform the edit on the related files in the FsdServer folder using WinHex and copied it back to the folder. But you have to first stop and the re-starting the AfwFsdSrv service first because apparently when this application is active the folder is being locked by the program.

Conversion of MOPS Display to OsiSoft’s PI Vision Display

Each of the MOPS Display (*.MCD files) is in the form of storage document which contains script that will be used by WinMOPS UI to display the screen. This part of storage can be copied using a hex editor such as WinHex and can be saved as text file.

Typical script as folows:

The PI Vision UI screen information is stored in the PI Vision’s web application database in SQL server. The database name is PIVision and the related table is dbo.StandardDisplays. This is based on the DisplayId and the description of each id is stored in EditorDisplay field. This field also contain the necessary UI components to be displayed in the PI Vision web application. It is text based and using JSON object notation.

So, it is possible to process the MOPS display script and output it into PI Vision’s JSON display script, then perform manual update to the table.

In the course of creating the parser and processing program, I sometimes encounter some error due to improper form of generated JSON display script that causes the PI Vision web will display “This display is not available” error as follow:

But this error can be easily resolved using the Event Viewer as follows:

Then the script generator program can be revised to conform to the desired format notation so that it can be processed by the PI Vision display routines.

For the text or rectangle object, this can be easily generated, but special treatment should be done for PI Point display, because although it has a screen object, but the necessary information for connecting the object to the PI point data is handled in different location.

The necessary information to connect PI Vision numeric data object to associated PI Point is stored in BaseDisplays table field called COG. It is in the XML format that will ties between symbol names and the PI Point data.

The script generator then can be programmed to emit the necessary COG XML string, and the generated XML then can be manually updated into the table.

There’s some limitation in the features of PI Vision displays, so that the generated JSON object may still be modified manually. But this is certainly can speed up the screen conversion process because much of the display components is already generated rather than keyed-in manually.

Experiences in Converting WMF to OsiSoft’s SVG Format

One of my projects involves some task to convert existing WMF file to SVG file to be used in OsiSoft’s PI Vision web application. After some exploration on the web, I decided to test the open source application in Java called wmf2svg from github.

After downloading and a successful compilation, perform command as follows:

But let’s test it using the WMF in the Apache Batik Framework file:

It is a success. Hmm, fishy isn’t it ? 🙂

So, I decided to perform remark of this piece of checking to see what would happen:

The type value is 0x2 which means disk according to the WMF specification and I don’t see why it is different from memory type. Now I run the above command that failed, and it is a success.

The SVG reveals tons of polygon primitives and when I viewed using a browser, it looked like this:

Note the white rectangular object inside the tube. The wmf2svg seems can’t process that unique object, but using Apache Batik’s WMFTranscoder routine that I created inside D:\Projects\BWMF\myWMF, it is as follows:

The object not get processed in wmf2svg reveals to be some pattern fill inside the object.

Where the original one from WMF is like this:

After some close examination of the content of generated SVG, I realized that those tons of polygon primitives is used to emulate the linear gradient inside the groups of polygon objects. When viewed using windows application, it appears to be smooth, but not in the case when viewed using the browser.

So, the task is to reduce the polygon primitives to retain just the one that creates the basic polygon objects and for the filling, I can use the LinearGradient based on the colors already given by the generated SVG. For the pattern fill object, I can import it from the SVG generated by WMFTranscoder. After some editing, the result look like this:

You can see that in OsiSoft’s PI Vision, the rectangular object (one marked with red box) inside the tube is black.

This is because the edited SVG is using the pattern to fill the object, and the browser can render it without any problem:

Seems that current SVG renderer in OsiSoft’s PI Vision still can’t handle the pattern fill in SVG. More investigation can be carried out on this issue, but I think that should be enough by now.

Exploring Apache Batik

I have some project that has the issue of converting the existing WMF files to the SVG. There are several services out there for converting these files to SVG but unfortunately when I try to download it asks for some price, where other services failed to recognize the file.

Then, I found that there are the open source application called Apache Batik that handles SVG and possibly can perform conversion of WMF to SVG. So, I decided to give it a try by downloading the binary version 1.10 to to D:\Projects\Batik folder.

Then I activate the program as follows:

But get the error as follows:

To peform close examination of the above error, I try to setup existing sources using NetBeans by just opening the copied source in D:\Projects\Batik\src. NetBeans is already familiar with this source structure.

The batik-squiggle.jar’s source code is in here (in red box):

There’s a source packages that eventually contains the Main.java for the start program.

Then perform debugging using this command:

Then, as usual, perform setting of the “Source Root” by adding the one with red box below:

Then the independent Java runtime sources can be opened using a plain “Open File”. Let’s first examing the causes of java.lang.NoClassDefFoundError for org/apache/batik/w3c/dom/ElementTraversal when running the program.

So, let’s open the NoClassDefFoundError.java in the java sources that is already copied to d:\projects\netbeans\Debug\src folder using NetBeans and perform placement of breakpoint at these locations:

Now in debug session the content of string is org/apache/batik/w3c/dom/ElementTraversal:

Using the 7-Zip to examine the existing batik-squiggle.jar reveals that it is indeed no class named org/apache/batik/w3c/dom/ElementTraversal. By searching the existing JAR file in the folder, it is found that the class is batik-all-1.10.jar.

Using the procmon, it is verified indeed that the batik-all-1.10.jar is not get included in the application when it is started.

So, this requires some modifications inside the batik-squiggle.jar. Using jar executable, perform extract as follows:

After the above command, the content of folder “Temp” folder will be:

Remove the copied batik-squiggle.jar so that it is not get processed by subsequent jar operation. Then create the required class path for the placement of ElementTraversal.class and copied it into the folder as follow:

Then I perform the command below to create a new jar file:

Please pay attention of the . (dot) at the end of the command. It is already causes me some hours to fiddle this command to make it right due to this missing dot 🙂

The above class not found error is now gone, but another error called java.lang.NoClassDefFoundError: org/apache/batik/w3c/dom/Window is occurred. So, repeat the above step.

After some steps in modifying the JAR file, the NoClassDefFoundError is now gone. The program GUI is now active, but seems there’s another error, so let’s collect this one in to the log file:

Now I have java.security.AccessControlException: access denied (“java.net.NetPermission” “setDefaultAuthenticator”) and Exception in thread “AWT-EventQueue-0” java.security.AccessControlException: access denied (“java.util.PropertyPermission” “user.dir” “read”).

In the JAR file there’s the file called svgbrowser.policy, let’s examine that and I have:

Since my JAR file is renamed to batik-squiggle.jar, the policy apply some strict rule that the JAR file name should be batik-squiggle-1.10.jar. So, let’s rename it and run the JAR file again. This time the Apache Batik Squiggle program runs without any glitches:

This concludes the preliminary observation of the Apache Batik Framework.

Exploration of GoLang in Windows Environment

The file go1.11.1.windows-386.zip is copied to d:\projects\go. Then perform set environment variable GOPATH to d:\projects\go.

Then perform compilation of the simple program of HelloWorld.go:

But it failed to create a proper executable file because the Sophos anti virus program recognized it as virus/spyware Mal/Zbot-FG:

I hope this is a false positive, so try to disable the anti virus for the moment, and I have:

The size of nearly 1 MB is rather bloated, maybe it contains debugging information and many static routines needed by the compiled program.

Try to perform debugging with WinDBG and it is totally blind to the existing debugging symbols. But using IDA Disassembler reveals tons of the runtime symbols information:

There’s a debugger for this kind of beast called Delve, but I doubt whether it has a capability to perform memory read/write breakpoint in Windows environment. There’s also a gdb debugger but according to the official go website it is unreliable.

So, a combination of WinDBG for debugging session and IDA Disassembler for an important information about debugging information will be handy for Windows environment.

For locating the main function, I can use IDA to locate the address which is:

Since the image base is located at 0x400000, then I can perform calculation 0x443550 – 0x400000, that in WinDBG it should be image00400000+0x43550.

In windows, the implementation for print function in the helloworld.go sample program is using kernel32’s WriteFile which calls to WriteConsoleA.

The runtime implementation for native OS calls is using runtime_asmstdcall inside go source sys_windows_386.s’. runtime·asmstdcall. There’s ASCII hex 0xB7 (183) between the word “runtime” and “asmstdcall”. This construct will not be found in text search using runtime_asmstdcall into the source in Windows OS. Why this hideous and peculiar construct ? Only the expert of GoLang developer can answer.

On the assembly instruction side, go seems to use its own style of instruction which at first creates confusion (or is it deliberately to create confusion ?), for example “MOVL CX, AX” which is actually “mov eax, ecx” in the actual assembly instruction on Windows OS.

This concludes the preliminary exploration of GoLang in Windows OS.

Some update: using 64 bit version of go compiler (go1.11.1.windows-amd64) will solve the Sophos’ Mal/Zbot-FG false positive virus problem.

Analysis of OsiSoft’s PI Base Subsystem

One of the PI Base Subsystem (pibasess.exe) tasks is responsible for handling PI Points tag data access from PI SMT. If this service is not running, when the user tries to get PI Points or tag information from PI SMT, it will get the error message as follows:

This article will discuss how this service is processing tag data called PI Points and how it access and perform searching of the PI Points.

The PI points data is recorded in the file called pipoints.dat. The structure and record information of this file can be viewed using pidiag.exe as follows:

The typical output as follows:

The file consist of directory data and the actual data. Directory data contains offset location and the record size. The location of directory data is determined by directory location information above. In this case, it is at offset 1024 on the file. Here is the typical directory information:

Let’s perform close examination of the directory record at offset 0x410. It contains a number to determine the offset location within the file for the data record and its size. Since it is Big Endian format, the number is 0x1163, and the data record size is 0x1A.

To determine the offset location and actual record size, multiply the number with 8 and the record size also with 8. So, the offset location will be 0x1163 * 0x8 = 0x8B18 and the record size will be 0x1A * 0x8 = 0xD0. Here is the partial record information of the calculated sample:

To facilitate the record access, the directory data is divided into blocks of 4096 (0x1000) size, but at the last block it is less than 4096 bytes. So, the PI points directory size above will consists of 6 blocks + extra blocks with size less than 4096, with the calculation as follows:

Since each directory record has 0x8 bytes size, so the maximum record number will be 3423.

Each block of 4096 bytes has the capacity to contain 4096 / 8 = 512 (0x200) records.

So, to access the data record, the PI program is using some number that will identify the block number and offset location of the data record.

To find a record, the number is divided by 512 to get the block # and perform and (&) operation with 0x1FF

For example the number 0x67E = 1662 / 512 = 3.24609375 (block #4 – counting start from 1). Then to find record location = 0x67E And 0x1FF = 0x7E = 126. This is the record number, and should be multiplied by 0x8 (0x7e * 0x8 = 0x3F0) to get the position of directory entry.

In this case, the directory entry is 0xF36B. To get the actual offset location of record, multiply this value with 0x8 = 0x79B58. This offset will points to the actual data record.

To facilitate the searching purposes, for example the tag name searching, all of the tag name string is arranged in some form of binary tree, and the string to be searched is compared by traversing the binary tree structure, starting from the root.

To determine which child node should be traversed, it is determined by the value of the string comparison. If it is less than zero, it will traverse to the right, if positive or zero it will traverse to the left. The zero value will indicates that the string to be searched is matched or found.

Here is the sample of the structure as it is viewed from memory:

As you can see from the above memory representation, each tree structure data contains the record number at offset 0x28, and string representation of tag name for the purpose of comparison process. The left child of the node is located at offset 0x0 and right child node is at 0x10.

For the above sample, record 0x8FC is the record for tag name LP7MB24-K57MS022_DOC01, 0x448 is the record for tag name LP7MB24-K57FV4082_ZT. The pibasess.exe will traverse this structure to try to find a match based on search criteria.

This concludes the PI points data structures and searching mechanisms of PI Points data in the pibasess.exe (PI Base Subsystem server).

What Would Happen When OsiSoft’s License is Expired ?

When the license is expired, upon activating the PI License Manager (pilicmgr.exe), it will show error in the message log:

Then all related core subsystems will also be restricted:

The user can not use the PI SMT because port 5450 in PI Network Manager (pinetmgr.exe) will not be activated by PI Base Subsystem.

You can use some keywords or sentences in the log file to find indication of possible license related problem, such as License Expiration Error, License Expired! – some operations could be restricted, reason: License has expired, Archive access is disabled due to license expiration.

Analysis of OsiSoft’s PI License Mechanism

The licensing mechanism in OsiSoft’s PI is governed by a service called “PI License Manager” or pilicmgr.exe. This application can be activated as a service or a command line program.

If it is invoked without a parameter, it will behave as a service and if it is invoked via command line, it will perform the task based on given parameter.

Let’s perform some command line task on this application:

One of the important information is the license expiration date above which determine how long all the PI application is active before the user should by the program.

You can see that the pilicmgr.exe will use the pilicense.dat file for its licensing information.

Here is the portion of the pilicense.dat viewed using hex editor such as WinHex:

You can see that is already in encrypted format, but how strong is the integrity of the security method ?

To answer the above question, I decided to perform detailed analysis of this application, especially when it tries to parse, decrypt and display the licensing information above.

Then, I arrived at the piece of data that will be used by pilicmgr.exe to process the licensing information, in decrypted format as follow:

The bytes portion in red color which is 0x5b7c6130 or 1534878000 in decimal format is the license expiration date in the form of the stamp. This data can be decoded into the actual date using the epoch converter website (https://www.epochconverter.com) and I will get Wednesday, August 22, 2018 2:00:00 AM GMT+07:00.

You can see this is in accordance with the license expiration date as shown by pilicmgr.exe program.

Then I perform an in depth analysis for assessment of how strong the security mechanism involved in encryption and decryption process. At first, I presumed that it will use the key pair mechanism, so although I know how to perform decryption, I can’t possibly perform encryption of the data.

But, I was surprised that security mechanism is very weak, so that I can perform decryption AND also encryption process. So, it is possible to perform modification of the data, such as license expiration date, perform encryption and let the pilicmgr.exe use the modified file.

The fact that I can still used the modified data also indicates the application do not use the checksum mechanism as one of the security measure.

So, I hope that OsiSoft PI personnel, after acknowledged this issue, will perform some improvement on its next version. I think that’s enough for now.

Analysis of OsiSoft’s PI SMT (System Management Tools)

This article will discuss how the SMT perform access to OsiSoft’s PI Point in the Current Values Plugin. This plug-in is used to view the real time current value of selected PI Points.

This plug-in is governed by a module in OSIsoft.SMT.Plugins.PICurrentValue.dll file. So, when the user clicks the search button after entering the proper tag name for tag selection, the module will call pisdk!CPIPoints::AddPIPointsInternal function to actually retrieves the PI Points.

This is the implementation of IPIPointsPrivate of AddPIPointsInternal in pisdk.dll which has IDL definition prototype as follows:

The psa2DAttrArray structure is array of retrieved PI Points structure. Let’s examine it:

As you can see, the structure contains header information and one or more records, depending on tag query result.

But how the SMT client connects to the server to retrieved the PI Points data ? Connection is implemented using PI Service called PI Network Manager. This module is executed as windows service. The executable name for this service is pinetmgr.exe.

For the purpose with communication with the client, the pinetmgr.exe will create a named pipe, for example in my machine is called PI3\Local:Pipe.

The client will then perform connection to this pipe using CreateFile in Kernel32.dll module. Then sending and receiving is done by using WriteFile and ReadFile respectively.

The data exchange is using Windows SSPI (Security Support Provider Interface) API encryption/decryption method.

Here you can see that the SMT will perform the communication with PI Network Manager using the given pipe name above. If the pinetmgr.exe service is not active, it will give error message as follows:

Error: Pinetmgr service is not running and caller has insufficient privileges to start it. Detected in PISDK subroutine New(). Application Exiting.

But again, where the actual PI point data came from ? This can be done by examining the pinetmgr.exe service.

When the pinetmgr received data from the client’s named pipe, it will try to communicate with PI Data Archive Server using port 5450. What program that is listening on port 5450 in PI Data Archive Server ? Also pinetmgr.exe.

The data is in encrypted format. Then it will receive the data, also in encrypted format to be forwarded to client’s pipe.

So, you can see that pinetmgr.exe can have a dual role, one as a client and one as a server that listen on port 5450. Now there arise the question, how to setup the pinetmgr.exe that can act as a server ?

To examine the above question, the pinetmgr.exe is copied to a testing server, then perform create service as follows:

Then perform the start service, but on detailed examination, the program still behaves as a client. Try to perform registry copy of HKEY_LOCAL_MACHINE\SOFTWARE\PISystem\PI from source server to testing server, re-start the pinetmgr.exe, but I have an error in Event Viewer as follows: Error locating rendezvous path.

This error is rather obscure and without the given debug symbol, it will be difficult to check for the reason of the above error.

The pinetmgr.exe service seems can be run at command line:

But performing the debugging using WinDBG do not arrive at the above error, but instead, it has different error message in event viewer which says Error 1063: Unable to start Service Control Dispatcher.

This is caused by the service tries to call the function StartServiceCtrlDispatcher and this function is failed. Let’s check why this is so ?

This is caused by some routines inside pinetmgr.exe that tries to call StartServiceCtrlDispatcher with the assumption it is called from the Service Control Manager.

So there’s no other way than to perform debugging inside the SCM, but several steps should be done before it is possible.

Perform the addition of registry values ServicesPipeTimeout and WaitToKillServiceTimeout inside HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control as follows:

Then inside the HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options perform addition of pinetmgr.exe key with content as follows:

So when the service is started, it will call the WinDBG that waits in the TCP port. Then using another WinDBG to try to connect to this port to perform debugging process.

And in this fashion, I notice that the error come after it is processing pisubsys.cfg. By using procmon.exe it is found that the service tries to find this file in the wrong location. This can be fixed by updating the InstallationPath key in HKEY_LOCAL_MACHINE\SOFTWARE\PISystem\PI.

After this, there are no service error exception, but I notice that the service is still not listening in the intended port of 5450.

After performing close examination of the source server that runs pinetmgr.exe, I notice that there are different size of pinetmgr.exe inside the PI and PIPC folder. Let’s perform delete of current create service and try to create the service from the one inside PI folder:

But still, the pinetmgr.exe service is not listening on port 5450. Why ?

Later on, I came across the link from OsiSoft’s web side https://techsupport.osisoft.com/Troubleshooting/KB/KB00685, with detail that On startup, pinetmgr will not listen to port 5450 until all PI core subsystems are fully started and their RPCs published with pinetmgr. In other words, the other subsystems must start and tell pinetmgr that they have started. In addition, PI Archive Subsystem (piarchss) will wait for PI Snapshot Subsystem (pisnapss) to fully start before publishing its RPC. However, it does not wait indefinitely. It waits for 30 minutes and if pisnapss is still not running, piarchss will shut itself down. If pinetmgr thinks piarchss was not started successfully, then pinetmgr will not accept any connections.

You can find list of PI core subsystems by searching through the net using “PI Data Archive core subsystems” keyword.

So, I perform service registration on remaining PI core subsystems, and then tries to start each service, but I found the service error as follows:

Failed to load tables for snapshot from C:\Program Files\PI\dat\piarcmem.dat. [2] The system cannot find the file specified.

Message ID: 2126

Fatal error in PI subsystem pisnapss: Failed to load archive memory tables, status: [2] The system cannot find the file specified.

Using procmon.exe, it is found that pisnapss service is trying to load non existent piarcmem.dat file in DAT folder. This file should be copied from the source server, but since it is used by the running service in production environment and can’t be stopped, I use hobocopy utility to perform copying process.

The hobocopy.exe installation file I used is hobocopy-unstable-64bit-20110505-01. Then I perform copy command as follows:

Then there are another error in pibasess subsystem which says Fatal error in PI subsystem pibasess: Failed to load PI configuration database, status: [2] The system cannot find the file specified. And also Fatal error in PI subsystem pibasess: Failed to load Acl Table, status: [2] The system cannot find the file specified.

By monitoring procmon.exe for missing files and perform shadow copy of the required files, the pibasess service is now can be executed without any error or exception.

Then I came across a rather cryptic error which says Fatal error in PI subsystem piarchss: Archive Point Count: 3110, Point Database count: 3112, status: [-11162] Archive and Base Point Count Mismatch – Contact Technical Support.

This is because the piarchss subsystem tries to perform comparison between piarcmem.dat and pipoints.dat and if the record count is not the same, it will throws the above error.

The solution is just make sure that all the piarcmem*.dat is copied from the source server to destination.

Then all core subsystems can be activated successfully and the pinetmgr service is now can listen on 5450 port:

Powered by WordPress and Bootstrap4