Analysis of OsiSoft’s PI Base Subsystem

One of the PI Base Subsystem (pibasess.exe) tasks is responsible for handling PI Points tag data access from PI SMT. If this service is not running, when the user tries to get PI Points or tag information from PI SMT, it will get the error message as follows:

This article will discuss how this service is processing tag data called PI Points and how it access and perform searching of the PI Points.

The PI points data is recorded in the file called pipoints.dat. The structure and record information of this file can be viewed using pidiag.exe as follows:

The typical output as follows:

The file consist of directory data and the actual data. Directory data contains offset location and the record size. The location of directory data is determined by directory location information above. In this case, it is at offset 1024 on the file. Here is the typical directory information:

Let’s perform close examination of the directory record at offset 0x410. It contains a number to determine the offset location within the file for the data record and its size. Since it is Big Endian format, the number is 0x1163, and the data record size is 0x1A.

To determine the offset location and actual record size, multiply the number with 8 and the record size also with 8. So, the offset location will be 0x1163 * 0x8 = 0x8B18 and the record size will be 0x1A * 0x8 = 0xD0. Here is the partial record information of the calculated sample:

To facilitate the record access, the directory data is divided into blocks of 4096 (0x1000) size, but at the last block it is less than 4096 bytes. So, the PI points directory size above will consists of 6 blocks + extra blocks with size less than 4096, with the calculation as follows:

Since each directory record has 0x8 bytes size, so the maximum record number will be 3423.

Each block of 4096 bytes has the capacity to contain 4096 / 8 = 512 (0x200) records.

So, to access the data record, the PI program is using some number that will identify the block number and offset location of the data record.

To find a record, the number is divided by 512 to get the block # and perform and (&) operation with 0x1FF

For example the number 0x67E = 1662 / 512 = 3.24609375 (block #4 – counting start from 1). Then to find record location = 0x67E And 0x1FF = 0x7E = 126. This is the record number, and should be multiplied by 0x8 (0x7e * 0x8 = 0x3F0) to get the position of directory entry.

In this case, the directory entry is 0xF36B. To get the actual offset location of record, multiply this value with 0x8 = 0x79B58. This offset will points to the actual data record.

To facilitate the searching purposes, for example the tag name searching, all of the tag name string is arranged in some form of binary tree, and the string to be searched is compared by traversing the binary tree structure, starting from the root.

To determine which child node should be traversed, it is determined by the value of the string comparison. If it is less than zero, it will traverse to the right, if positive or zero it will traverse to the left. The zero value will indicates that the string to be searched is matched or found.

Here is the sample of the structure as it is viewed from memory:

As you can see from the above memory representation, each tree structure data contains the record number at offset 0x28, and string representation of tag name for the purpose of comparison process. The left child of the node is located at offset 0x0 and right child node is at 0x10.

For the above sample, record 0x8FC is the record for tag name LP7MB24-K57MS022_DOC01, 0x448 is the record for tag name LP7MB24-K57FV4082_ZT. The pibasess.exe will traverse this structure to try to find a match based on search criteria.

This concludes the PI points data structures and searching mechanisms of PI Points data in the pibasess.exe (PI Base Subsystem server).

What Would Happen When OsiSoft’s License is Expired ?

When the license is expired, upon activating the PI License Manager (pilicmgr.exe), it will show error in the message log:

Then all related core subsystems will also be restricted:

The user can not use the PI SMT because port 5450 in PI Network Manager (pinetmgr.exe) will not be activated by PI Base Subsystem.

You can use some keywords or sentences in the log file to find indication of possible license related problem, such as License Expiration Error, License Expired! – some operations could be restricted, reason: License has expired, Archive access is disabled due to license expiration.

Analysis of OsiSoft’s PI License Mechanism

The licensing mechanism in OsiSoft’s PI is governed by a service called “PI License Manager” or pilicmgr.exe. This application can be activated as a service or a command line program.

If it is invoked without a parameter, it will behave as a service and if it is invoked via command line, it will perform the task based on given parameter.

Let’s perform some command line task on this application:

One of the important information is the license expiration date above which determine how long all the PI application is active before the user should by the program.

You can see that the pilicmgr.exe will use the pilicense.dat file for its licensing information.

Here is the portion of the pilicense.dat viewed using hex editor such as WinHex:

You can see that is already in encrypted format, but how strong is the integrity of the security method ?

To answer the above question, I decided to perform detailed analysis of this application, especially when it tries to parse, decrypt and display the licensing information above.

Then, I arrived at the piece of data that will be used by pilicmgr.exe to process the licensing information, in decrypted format as follow:

The bytes portion in red color which is 0x5b7c6130 or 1534878000 in decimal format is the license expiration date in the form of the stamp. This data can be decoded into the actual date using the epoch converter website (https://www.epochconverter.com) and I will get Wednesday, August 22, 2018 2:00:00 AM GMT+07:00.

You can see this is in accordance with the license expiration date as shown by pilicmgr.exe program.

Then I perform an in depth analysis for assessment of how strong the security mechanism involved in encryption and decryption process. At first, I presumed that it will use the key pair mechanism, so although I know how to perform decryption, I can’t possibly perform encryption of the data.

But, I was surprised that security mechanism is very weak, so that I can perform decryption AND also encryption process. So, it is possible to perform modification of the data, such as license expiration date, perform encryption and let the pilicmgr.exe use the modified file.

The fact that I can still used the modified data also indicates the application do not use the checksum mechanism as one of the security measure.

So, I hope that OsiSoft PI personnel, after acknowledged this issue, will perform some improvement on its next version. I think that’s enough for now.

Analysis of OsiSoft’s PI SMT (System Management Tools)

This article will discuss how the SMT perform access to OsiSoft’s PI Point in the Current Values Plugin. This plug-in is used to view the real time current value of selected PI Points.

This plug-in is governed by a module in OSIsoft.SMT.Plugins.PICurrentValue.dll file. So, when the user clicks the search button after entering the proper tag name for tag selection, the module will call pisdk!CPIPoints::AddPIPointsInternal function to actually retrieves the PI Points.

This is the implementation of IPIPointsPrivate of AddPIPointsInternal in pisdk.dll which has IDL definition prototype as follows:

The psa2DAttrArray structure is array of retrieved PI Points structure. Let’s examine it:

As you can see, the structure contains header information and one or more records, depending on tag query result.

But how the SMT client connects to the server to retrieved the PI Points data ? Connection is implemented using PI Service called PI Network Manager. This module is executed as windows service. The executable name for this service is pinetmgr.exe.

For the purpose with communication with the client, the pinetmgr.exe will create a named pipe, for example in my machine is called PI3\Local:Pipe.

The client will then perform connection to this pipe using CreateFile in Kernel32.dll module. Then sending and receiving is done by using WriteFile and ReadFile respectively.

The data exchange is using Windows SSPI (Security Support Provider Interface) API encryption/decryption method.

Here you can see that the SMT will perform the communication with PI Network Manager using the given pipe name above. If the pinetmgr.exe service is not active, it will give error message as follows:

Error: Pinetmgr service is not running and caller has insufficient privileges to start it. Detected in PISDK subroutine New(). Application Exiting.

But again, where the actual PI point data came from ? This can be done by examining the pinetmgr.exe service.

When the pinetmgr received data from the client’s named pipe, it will try to communicate with PI Data Archive Server using port 5450. What program that is listening on port 5450 in PI Data Archive Server ? Also pinetmgr.exe.

The data is in encrypted format. Then it will receive the data, also in encrypted format to be forwarded to client’s pipe.

So, you can see that pinetmgr.exe can have a dual role, one as a client and one as a server that listen on port 5450. Now there arise the question, how to setup the pinetmgr.exe that can act as a server ?

To examine the above question, the pinetmgr.exe is copied to a testing server, then perform create service as follows:

Then perform the start service, but on detailed examination, the program still behaves as a client. Try to perform registry copy of HKEY_LOCAL_MACHINE\SOFTWARE\PISystem\PI from source server to testing server, re-start the pinetmgr.exe, but I have an error in Event Viewer as follows: Error locating rendezvous path.

This error is rather obscure and without the given debug symbol, it will be difficult to check for the reason of the above error.

The pinetmgr.exe service seems can be run at command line:

But performing the debugging using WinDBG do not arrive at the above error, but instead, it has different error message in event viewer which says Error 1063: Unable to start Service Control Dispatcher.

This is caused by the service tries to call the function StartServiceCtrlDispatcher and this function is failed. Let’s check why this is so ?

This is caused by some routines inside pinetmgr.exe that tries to call StartServiceCtrlDispatcher with the assumption it is called from the Service Control Manager.

So there’s no other way than to perform debugging inside the SCM, but several steps should be done before it is possible.

Perform the addition of registry values ServicesPipeTimeout and WaitToKillServiceTimeout inside HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control as follows:

Then inside the HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options perform addition of pinetmgr.exe key with content as follows:

So when the service is started, it will call the WinDBG that waits in the TCP port. Then using another WinDBG to try to connect to this port to perform debugging process.

And in this fashion, I notice that the error come after it is processing pisubsys.cfg. By using procmon.exe it is found that the service tries to find this file in the wrong location. This can be fixed by updating the InstallationPath key in HKEY_LOCAL_MACHINE\SOFTWARE\PISystem\PI.

After this, there are no service error exception, but I notice that the service is still not listening in the intended port of 5450.

After performing close examination of the source server that runs pinetmgr.exe, I notice that there are different size of pinetmgr.exe inside the PI and PIPC folder. Let’s perform delete of current create service and try to create the service from the one inside PI folder:

But still, the pinetmgr.exe service is not listening on port 5450. Why ?

Later on, I came across the link from OsiSoft’s web side https://techsupport.osisoft.com/Troubleshooting/KB/KB00685, with detail that On startup, pinetmgr will not listen to port 5450 until all PI core subsystems are fully started and their RPCs published with pinetmgr. In other words, the other subsystems must start and tell pinetmgr that they have started. In addition, PI Archive Subsystem (piarchss) will wait for PI Snapshot Subsystem (pisnapss) to fully start before publishing its RPC. However, it does not wait indefinitely. It waits for 30 minutes and if pisnapss is still not running, piarchss will shut itself down. If pinetmgr thinks piarchss was not started successfully, then pinetmgr will not accept any connections.

You can find list of PI core subsystems by searching through the net using “PI Data Archive core subsystems” keyword.

So, I perform service registration on remaining PI core subsystems, and then tries to start each service, but I found the service error as follows:

Failed to load tables for snapshot from C:\Program Files\PI\dat\piarcmem.dat. [2] The system cannot find the file specified.

Message ID: 2126

Fatal error in PI subsystem pisnapss: Failed to load archive memory tables, status: [2] The system cannot find the file specified.

Using procmon.exe, it is found that pisnapss service is trying to load non existent piarcmem.dat file in DAT folder. This file should be copied from the source server, but since it is used by the running service in production environment and can’t be stopped, I use hobocopy utility to perform copying process.

The hobocopy.exe installation file I used is hobocopy-unstable-64bit-20110505-01. Then I perform copy command as follows:

Then there are another error in pibasess subsystem which says Fatal error in PI subsystem pibasess: Failed to load PI configuration database, status: [2] The system cannot find the file specified. And also Fatal error in PI subsystem pibasess: Failed to load Acl Table, status: [2] The system cannot find the file specified.

By monitoring procmon.exe for missing files and perform shadow copy of the required files, the pibasess service is now can be executed without any error or exception.

Then I came across a rather cryptic error which says Fatal error in PI subsystem piarchss: Archive Point Count: 3110, Point Database count: 3112, status: [-11162] Archive and Base Point Count Mismatch – Contact Technical Support.

This is because the piarchss subsystem tries to perform comparison between piarcmem.dat and pipoints.dat and if the record count is not the same, it will throws the above error.

The solution is just make sure that all the piarcmem*.dat is copied from the source server to destination.

Then all core subsystems can be activated successfully and the pinetmgr service is now can listen on 5450 port:

Analysis of OsiSoft’s PI Vision Web Application (Part 2)

When I try to perform screen design by clicking on one of the PI Vision pages I’ve designed, I get an error message as follows:

Timestamp: ‎5‎/‎3‎/‎2018‎ ‎10‎:‎46‎:‎18‎ ‎AM
Severity: Error
Message: Data Error
Details: Could not load file or assembly ‘OSIsoft.PI.Net.Core, Version=1.7.6.0, Culture=neutral, PublicKeyToken=c3309f0734ba2805’ or one of its dependencies. The system cannot find the file specified.

After trying to copy the missing file from the original server, the error still there.

Trying to monitor it using ProcessMonitor (Procmon.exe) reveals as follows:

Checking into destination folder reveals that it has a different version called v4.0_1.8.3.0__c3309f0734ba2805. So, why it insists on loading non-existing 1.7.6.0 version ?

Using ILSpy, it is revealed that the required version is referenced from assembly called OSIsoft.AFSDK:

And in version information, I can see that it is indeed version 1.7.6.0 that is required:

Get an updated version of OSIsoft.AFSDK.dll from source to destination, and seems that the web application can get the correct version now:

Let’s whether the error is recurring again by try to perform design, the previous error is solved.

But another came which is Data Error : Could not load file or assembly ‘OSIsoft.PI.Configuration, Version=1.8.3.0, Culture=neutral, PublicKeyToken=c3309f0734ba2805’ or one of its dependencies. The system cannot find the file specified.

Perform replace of entire GAC_MSIL folder from source to destination. Let’s try again.

But now I have another error which is Data Error : Retrieving the COM class factory for component with CLSID {7D36BED3-9635-484F-92E0-478369C450EC} failed due to the following error: 80040154 Class not registered (Exception from HRESULT: 0x80040154 (REGDB_E_CLASSNOTREG)).

Let’s find out in the source server, what kind of the above component is that:

Perform copying of entire folder of PIPC from source server to destination server Then try to perform the COM registration as follows:

D:\Program Files (x86)\PIPC\pisdk>regsvr32 PISDKRegistry.dll

But I have an error the module “PISDKRegistry.dll” failed to load as follows:

Using Dependency Walker program I’ve found as follows:

Since this is the 32 bit application, copy the required file from C:\Windows\SysWOW64 in the source server to destination. After copying all of the required dependencies, now I have:

There are no errors on the PI Vision display now. But when try to click on the data, I have HTTP Error 503. The service is unavailable.

Checking on Event Viewer, I have a crashed w3wp with error below:

Application: w3wp.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Exception Info: System.AccessViolationException
at PISDKRegistry.ISDKServers.get_DefaultServer()
at OSIsoft.PI.Configuration.Schema.PISDKRegistrySchema+<>c__DisplayClass25_0.b__0(PISDKRegistry.IPISDKReg)
at OSIsoft.PI.Configuration.Schema.PISDKRegistrySchema.AccessRegistry(System.Action`1)
at OSIsoft.PI.Configuration.Schema.PISDKRegistrySchema.RunInReaderLock(System.Action`1)
at OSIsoft.PI.Configuration.Schema.PISDKRegistrySchema.get_DefaultServer()
at OSIsoft.PI.Configuration.PISDKRegistryDirectoryProvider.AddDefaultServiceToDomainTable(OSIsoft.PI.Configuration.DomainTable)
at OSIsoft.PI.Configuration.PISDKRegistryDirectoryProvider.GetDomainTable()
at OSIsoft.PI.Configuration.PISDKRegistryDirectoryProvider.GetTable[[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]](OSIsoft.PI.Configuration.TableType)

Then in the earlier event I get:

The description for Event ID 5 from source PISDK cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.

If the event originated on another computer, the display information had to be saved with the event.

The following information was included with the event:

Failed to create/open registry key – SOFTWARE\Wow6432Node\PISystem.

Perform registry export from source from indicated location and import it into destination. Next I have this error:

Failed to create/open Symbolic link to ..WOW6432Node\PISystem\PISDK

In the source server, perform examine of registry links using this command below:

The result is there’s a registry link HKLM\Software\PISystem\PI-SDK -> HKLM\SOFTWARE\Wow6432Node\PISystem\PI-SDK

Export the registry from source server for HKLM\Software\PISystem and HKLM\SOFTWARE\Wow6432Node\PISystem and import to destination server. Next is to remove the reference of PI-SDK in HKLM\Software\PISystem, since it is actually a registry key link (REG_LINK) to prevent error when try to create a link.

Then perform registry link using regln-x64 command below:

Now there’s no crashed w3wp.exe and the PI Vision pages can be shown properly. Next, I notice when I try to get the PI AF data, I get an error as follows:

Let’s examine it starting from the UI component that is involved to obtain the data. The above “Internal Server Error” string should contain some PI Asset data. The UI for this data is triggered by clicking on right arrow:

Using inspector, the UI element is called disclosure-indicator with method navIntoChild as follow :

The above element is part of PIVision\Scripts\app\common\breadcrumblist.html with script handler in PIVision\Scripts\app\common\PIVisualization.breadcrumblist-directive.js.

When the right arrow is clicked, it will execute navigate function as follows:

The navigate method is as follow:

The call to scope.getChildren will call to method getChildren method in PIVision\Scripts\app\editor\tools\PIVisualization.tool-search-directive.js. It will then call assetCache.getChildren.

The assetCache object is declared in PIVision\Scripts\app\editor\services\PIVisualization.asset-cache.js.

But first, let’s check the detail of object error in the event of processNavigationError method inside getChildren to see whether there are additional information for the error. But apart from “Internal Server Error” message, there are no other useful clue.

So, by examining getChildren method in assetCache source code, it is found that actually it calls to webServices.getChildAssets(parent.Path). And if you already read the previous article you will find that is is part of PIVision\Scripts\app\common\PIVisualization.web-services.js.

Inside this method it will translate to _callToWs with ‘Navigation/ChildAssets’ among one of its parameter. Then using network monitoring tool in Firefox, I will get information of the complete URL that the web service is trying to call which is http://172.xxx.yyy.zzz/PIVision/Navigation/ChildAssets?parentPath=af:\\xxx\yyy

So, when I try to use the above URL, surely there’s the HTTP 500 Internal server error.

Now let’s check the server side handling of this request, which is actually resides in OSIsoft.PIVisualization.Web.Controllers.NavigationController class. This class is compiled in the file called OSIsoft.PIVisualization.WebCommon.dll.

The ChildAssets is handled using method called GetChildAssets with method executions as follows:

From the above routines, you can see that if any of the error condition is fullfilled, it will emit HttpStatusCode.InternalServerError. But let’s check whether the retrieved list can give some helpful information regarding the error, i.e. from the red box above.

The method GetChildAssets is called using async method, so using WinDBG debugger, if you try to perform breakpoint to:

OSIsoft.PIVisualization.Web.Controllers.NavigationController.GetChildAssets

The actual calling method will be translated into something else, and in this case it is transformed into:

OSIsoft.PIVisualization.Web.Controllers.NavigationController+d__6.MoveNext

The CreateResults above is called using await method, and in the WinDBG debugger, it will call:

System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification

And by perform break point after this call inside this method:

The result task structure can be retrieved by querying the contain of rcx register after assigned by memory content of rbp-10h (see above). Task structure object as follows:

The task will contain:

System.Threading.Tasks.Task`1[[System.Collections.Generic.IList`1[[OSIsoft.PIVisualization.Services.Search.RestSearchResult, OSIsoft.PIVisualization.StorageService]], mscorlib]]

The returned list data is placed in object called m_result (see above), it will contain IList of type RestSearchResult:

The RestSearchResult is place inside object called _items:

It contains some data on first element, let’s check it:

The error information is stored in object called Error which has System.String type, let’s examine it:

In this way, I can find an underlying error which is not shown in PI Vision, which contains The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel.

After performing some more debugging session, it is found that the above error is originated from OSIsoft.Search.DefaultSearch.GetSearchResults. This method resides in OSIsoft.Search.dll.

As usual, since it is run as a task, it is transformed to OSIsoft.Search.DefaultSearch+d__22.MoveNext.

It will use the WebClient object to access this url below:

https://xxx.yyy.zzz.com/piwebapi/search/Children?parent=af:\\xxx\yyy&fields=Name;Description;ItemType;UniqueId;Template;HasChildren;Paths;Plottable&count=1000

But this time, perform testing of the above url using a browser proves to be no problem. Since it is using a WebClient, I try to create a very small ASP.NET web program that is using the above URL for access, and the result is an exception called System.Security.Authentication.AuthenticationException, with error description as The remote certificate is invalid according to the validation procedure.

Using the browser, there will be certificate error message This CA Root certificate is not trusted because it is not in the Trusted Root Certification Authorities store.

So, perform certificate export from source server using IIS in Server Certificates menu and import it into Trusted Root Certification Authorities. And perform WebClient testing again.

But this time, the testing returns The remote server returned an error: (401) Unauthorized. And also try to use the above link to access the PI Asset still returns HTTP 500 Internal server error.

Perform un-remark at near the end of of the file OSIsoft.REST.Host.exe.config in OsiSoft’s WebAPI folder to enable trace data to be written to PIWebAPI.log. Then perform service re-start and re-run the web service request.

The result of log as follows:

OSIsoft.REST.Host.exe Error: 0 : [2018-06-12T06:20:33.9035983Z] Level=Error, Kind=End, Category=’System.Net.Http.Formatting’, Id=af7ccae7-dab9-47db-aee9-870f899fe879, Operation=AFJsonMediaTypeFormatter.WriteToStreamAsync, Status=500 (InternalServerError), Exception=System.Web.Http.HttpResponseException: Processing of the HTTP request resulted in an exception. Please see the HTTP response returned by the ‘Response’ property of this exception for details.
at OSIsoft.REST.Formatters.AFJsonMediaTypeFormatter.WriteToStream(Type type, Object value, Stream stream, Encoding encoding)
at OSIsoft.REST.Formatters.AFJsonMediaTypeFormatter.WriteToStreamAsync(Type type, Object value, Stream stream, HttpContent content, TransportContext context, CancellationToken token)
at System.Net.Http.Formatting.MediaTypeFormatter.WriteToStreamAsync(Type type, Object value, Stream writeStream, HttpContent content, TransportContext transportContext)
at System.Web.Http.Tracing.Tracers.MediaTypeFormatterTracer.<>c__DisplayClassd.b__c()
at System.Web.Http.Tracing.ITraceWriterExtensions.TraceBeginEndAsync(ITraceWriter traceWriter, HttpRequestMessage request, String category, TraceLevel level, String operatorName, String operationName, Action`1 beginTrace, Func`1 execute, Action`1 endTrace, Action`1 errorTrace)

This is parallel with the error on client side by examining it in WinDBG with the error The remote server returned an error: (500) Internal Server Error. The CLR exception event filter in WinDBG must be activated after the call to GetChildAssets.

When the first exception is occured, perform series of command below to see the error message:

Although the above message log perform some information about the routine that caused the exception, it is not provide a useful clue of the exact problem. So by performing a debugging session on running OSIsoft.REST.Host.exe I arrive at the source of exception as follows:

Inside PIImpersonationContext routine, there’s some check whether the given identity is an Anonymous user and throws an exception:

Let’s perform check of the Identity given by PIIdentity at the call to PIImpersonationContext when the exception will occur:

You can see that the PI Web API server receives an identity of NT AUTHORITY\ANONYMOUS LOGON, and it will throws exception when it is Anonymous.

Unfortunately there’s no solution to this issue, because it is by design in part of PI Vision access to PI Web API using WebClient.

The consequences is that PI Vision web application can not be placed in separate server location. This is the part of routine that caused the issue below:

You can see from the above picture, that inside OSIsoft.Search.DefaultSearch.GetSearchResults, there’s direct call to OpenReadTaskAsync without any anticipation for some given client’s credential data passing through WebClient object.

If the object in question is outside of local server, the user will always be an Anonymous and the Web API will always throws exception.

This concludes the analysis session.

Analysis of OsiSoft’s PI Web API

Content of the Web API folder is placed to \\yyyy\d$\WebApp\PIWebApi. Then try to perform command line operation by invoking OSIsoft.REST.Host.exe with administrative privilege, but it exits as soon as it is get called.

Perform WinDBG and event filter for CLR Exception, it shows that the application throws DirectoryNotFoundException

And the source of the Exception is from InstallUtil.DataDirectory, which contains null value. By examining the source using ILSpy, the DataDirectory is retrieved from registry called HKLM\Software\PISystem\WebAPI.

Perform registry import from source server to destination. Now the server application runs without exit. But the question arise, how to access the server using web client ? In other words, what URL that the client required to be sent to the server ?

After some investigations, it is found that the application responds only to PIWebAPI url link. But when trying to access the link using browser, it gets only HTTP 500 error.

In the source server, the application is run inside service framework, let’s try this:

But as soon as the service is started, it is abruptly stopped. And show this error message:

Checking on Event Viewer I have:

Let’s try again in command line mode with administrative privilege and perform breakpoint at the System.Net.HttpListener.AddAllPrefixes method. There is no exception error.

Using ILSpy, the AddAllPrefixes method is actually using the unmanaged API call to HttpAddUrlToUrlGroup. The call is a success, but as I described above, it causes HTTP 500 error.

In the source server, I try to stop the service and run the application in command line mode, yielding the similar result, a blank page in a browser. So, seems that PI Web API can not run in command line mode. So I will leave it at that, and pursue the service framework instead.

Later on, I’ve found that the registered URL in HTTP Server in windows can be viewed using netsh command:

To see whether the service is creating the above URL registration link, I try to stop the service, and when the service is stopped, the registered URL is still there !

Later on, I’ve found that to perform URL registration can be done using netsh also:

But the error says Url reservation add failed, Error: 87 The parameter is incorrect.

Should be this one:

Run the application using service framework. This time there are no Access Denied run time error message, and in the browser at last I have:

This concludes the preliminary analysis of the application.

How to Solve OsiSoft’s PI System Management Tools (SMT) Error

When I try to activate PI SMT (System Management Tools), I found error as follows:

The error says:

Unable to load PI SDK. Creating an instance of the COM component with CLSID {3BCB2DC2-5F66-11d1-BD64-0060B0290178} from IClassFactory failed due to the following error: 8007000E Not enough storage is available to complete this operation. (Exception from HRESULT: 0x8007000E (E_OUTOFMEMORY)).

The above error message is rather misleading, because the given CLSID is from pisdk.dll which is already works just fine.

The actual error is when the SMTHost.exe is try to create an object instance from PISDKRegistry.dll via pisdk!SDKSessionMgr::GetSDKReg.

This is the typical message when the 32 bit application is trying to instantiate a class compiled in 64 bit architecture. So, perform repeat registration using regsvr32 from the correct 32 bit binaries:

Will make the SMT works again.

Case of Missing PI Builder Excel Addins

So, I’ve perform un-install and reinstall the PI Datalink due to license issue, but after installation is successful, I’ve found that PI Builder is missing:

Perform copy of PIAFBuilder.dll and it’s associated files to C:\Program Files (x86)\PIPC\Excel. This is for 32 bit version, since I’m using Excel 2007 which is still 32 bit version.

The registry location will be:

HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\Office\Excel\Addins

Create registry key named PIAFBuilder, and inside this key, then edit registry as follows:

And voila! The PI Builder menu is now shown:

Analysis of OsiSoft’s PI Vision Web Application

PI Vision web application can be accessed through:

https://172.xxx.yyy.zzz/pivision

Setting of directory in IIS 8.0 as follows:

For the purpose of analysis, the PIVision folder is copied to \\yyyyy\d$\WebApp. Then using IIS perform setup of Application Pool called PIPool and make PIVision folder as application.

Now when running through the browser I get:

Checking into Event Viewer there’s no information regarding this event. Perform the View Column selection in Task Manager to get more info about w3wp.exe to be debugged:

Now I can get which w3wp.exe to be debugged, but seems it is not run as X86 32 bit process because:

Could not attach to process nnnn, NTSTATUS 0xC00000BB.

Obtain and install 64 bit version of Windows Debugger (WinDBG) and now w3wp.exe can be debugged properly. First is try to set some event filters debugger to trap whether some interesting exceptions. But the page still shown as above without giving any exception reported.

When the web.config is set to a minimum information such as this:

The browser now will give some more information regarding the errors:

Now I can add more and more information to the web.config and see what tag name that caused the IIS curled up to 500 Internal server error:

When the above tag is added to the web.config, IIS will again curled up to 500 Internal server error which may cause consternation to the web application developer. So, remark it for a moment.

After the OSIsoft.AFSDK and its related assemblies is copied, I get:

Method not found: ‘!!0[] System.Array.Empty()’

With callstack as follows:

Install .NET Framework 4.6.2 NDP462-KB3151800-x86-x64-AllOS-ENU and restart server. Now I have:

Using the WinDBG debugger, perform loading of .NET Framework debugging utility sos.dll as follows:

And at the exception below:

Perform disassembly as follows:

To identify the call information of the above instruction, use !u command from sos.dll utility:

By performing some text search to the above address (orange text), I have:

Using ILSpy for related method (InitializeFeaturesAFServer) I have some interesting information as follows:

What if I try to add the “FeaturesServer” key to the AppSettings to the web.config ? But let’s try to verify it by perform break into:

this.featuresAFServer = serviceModule.AppSettings[“FeaturesServer”];

But it is nowhere to be found in the disassembled code in WinDBG through sos.dll. So try to perform break point to the start of function call:

OSIsoft.PBViewer.Web.PBVApplication.InitializeFeaturesAFServer()

Using this command seems not to work:

To perform workaround the above issue, I’ve downloaded clr.pdb symbol file and perform steps to perform breakpoint as follows:

Then after the module load event perform:

After this call, the JIT should already perform compilation of the method, so:

I will get:

Perform clear on the break point so that the debugger will not stopped at unnecessary InvokeMethod. And use the suggestion from Name2EE result, to be arrived at:

So, it is just calling the NameValueCollection class for “FeaturesServer”. But where did it store this setting ? By examining codes inside SaveSetting method, seems it is saved into database and table called Settting.

But let’s focus on the causes of the above error that causes null value in GetDefaultPISystem. After some debugging session, it is revealed that there are missing information in:

So, perform xml merging from the existing server to this location. Now I have:

401 – Unauthorized: Access is denied due to invalid credentials.

You do not have permission to view this directory or page using the credentials that you supplied.

This is because in OnEnter method in System.Web.Security.UrlAuthorizationModule there some checks as follows:

And the web.config for authorization process is determined as follows:

The error code 401 comes from ReportUrlAuthorizationFailure method and in the web.config, the authorization is set to deny all user, so the browser is supposed to perform some authentication process to the web server. The method of how the browser provide suitable authenticated identity is determined by the mode, and in this case it is using Windows credentials.

To configure it using http instead of https, just perform remark of the authorization tag above.

But let’s come back to when I try to add the rewrite tag that causes the HTTP 500 error. By checking the error code given by:

nativerd!SECTION_GROUP_TABLE::ParseConfigSetting

It is found to be 0x8007000d. After some searching thru the net, there’s indication that URL Rewrite module is not yet installed. So perform download of URL Rewrite module rewrite_amd64_en-US and installed into the machine.

After URL Rewrite module is installed, the rewrite tag can be added without causing HTTP 500 Internal server error anymore.

Now, the page is again showing 401.2.: Unauthorized: Logon failed due to server configuration error.

Perform remark on the authorization tag that denies all user, then perform modification on Authorization for the website to enable Windows Authentication.:

If the Windows Authentication is not shown, you can installed it using Server Manager, then Manage, then Add Roles and Features. But now if I try this url:

The application will re-direct it into:

Which will cause 404 – File or directory not found error. Add the user group called “PI Vision Users” with member as follows:

Now the front page can be shown:

But there are some errors as follows:

Communication Error: An unexpected error has occured.

Using the Inspector, the element is generated from:

Try to view the source in the web browser appears in bundle:

Putting alert(“x”) in \PIVision\Scripts\app\editor\layout\PIVisualization.nav-controller.js will shows the alert box. But on trying to examine the callstack, the function seems already minified.

Modifying the web.config debug flag to true:

Seems to disable the minification process:

The error is come from this callstack:

The error comes from loadFolderCacheHierarchy function inside \PIVision\Scripts\app\editor\start\PIVisualization.home-controller.js as follows:

Inside function loadFolderCacheHierarchy there’s a call to folderCache.loadHierarchy using a promise object. The folderCache object itself is injected using angular’s $inject and it is created in \PIVision\Scripts\app\editor\start\PIVisualization.folder-cache.js using angular’s factory method as follows:

The loadHierarchy itself is using webService method called getHierarchy as follows:

The webServices object itself is again injected via angular construction and getHierarchy method itself resides in PIVision\Scripts\app\common\PIVisualization.web-services.js.

This method is actually a wrapper that perform _callToWs by passing this url as follows:

http://172.xxx.xxx.xxx/PIVision/Navigation/FolderHierarchy?displayFolderId=-1&pbFolderId=-1

By performing url access via the browser, I get error as follows:

This error (HTTP 500 Internal Server Error) means that the website you are visiting had a server problem which prevented the webpage from displaying.

By performing breakpoint at:

OSIsoft.PIVisualization.Web.Controllers.NavigationController.GetFolderHierarchy

I arrive at:

Then I arrive at exception by enabling exception breakpoint at WinDBG:

By examining the SqlException object to get the message, I have the message as:

A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: SQL Network Interfaces, error: 26 – Error Locating Server/Instance Specified)

Checking on the server, it is found that SQL Server Browser Service is not running. Try to run it, but I have:

(provider: SQL Network Interfaces, error: 28 – Server doesn’t support requested protocol)

In the SQL Server Configuration Manager, perform revision as follows:

Test the direct web access to the above url again, this time there are no HTTP 500 error and also no error in the PI Vision web application.

Powered by WordPress and Bootstrap4