Document toolboxDocument toolbox

SA380 series Syslogs

Owned by Janos Vaczi (Unlicensed)
Last updated: Nov 12, 2014 by Former user (Deleted)
9 min read

Introduction

SA380 series loggers write a journal about the most important internal activities like startup, shutdown, communication.

The syslog is the first place to look during debugging and investigation.

Where to find the Syslog

SA380

GUI

Go to Menu. View Data. Select the appropriate date. Select the dateLog.txt file.

   

Syslog will show up as text file:

Web interface

 

You have to have network access to the logger to do this. It can be achieved by connecting directly with an Ethernet cable, or connecting remotely through a network.

You need to have the IP address of the logger!

 

Enter the logger's IP address into your web browser:

You will be prompted to enter username/password:

You will be able to browse the data stored on the logger as a folder:

Navigate to the syslog file:

                

The folder belonging an actual date can have multiple digital log files and a syslog file. Select the dateLog.txt syslog file:

 

SA380TX

GUI

Similarly to SA380: Go to Menu. Go to Digital instead of Real-time system log! Press Show. Select system log and press Show.

    

Syslog has to show up:

Web interface

For the SA380TX logger you have to enter the IP address and have to specify the port as well:

You will be prompted for username/password. This and navigating to the actual syslog is the same as on the SA380.

Notice that each syslog lines are broken into two lines on the screen to be able to view most of the parameters.

SA380TX-L

dPanel

This assumes you have connection to the SA380TX-L logger either connected with a USB cable or you have network access to the logger through GPRS alternatively there can be Ethernet card installed.

First you have to select the right type of connection:

For USB cable connection, select USB:

 dPanel have to pick up the logger connected to the computer. Press Connect.

The connected logger will enumerate the information on the Real-time tab:

To get to the syslog, select Historic:

 

Select Syslog on the sub-tab:

 Normally it shows the last syslog entries retrieved from the logger. First time around it will be empty.

To simply retrieve the latest syslog, press Refresh data:

Web interface

SA380TX-L loggers with new version of firmware (version number?) have web-based configuration tool. The presence of the web tool can be checked with dPanel:

 On the real-time tab we get a valid WWW version, which indicates the web-tool is installed on the logger.

Enter the logger's IP address into your browser:

The web configuration tool needs javascript enabled in your browser.

Successful connection will show the loading screen first:

Once loaded it shows the current configuration of the logger:

Navigate to the syslog: Select Historic tab. 

Historic data normally shows the syslog by default. If not, select Syslog sub-tab:

WebCFG retrieves the latest syslog entries:

 

How to read the Syslog

Basics

SyslogDescriptionExample
System startup                  First message after the logger boots.
System startup 1.5T444 29Oct14 serno=65532
System shutdown                 Usually the last message in the syslog before the logger powers down or restart.no parameters
Daily status                    

Available disk space on each disks.

Slave unit statuses.

GPRS and dialup status.

Daily status dsk1:374M dsk2:Miss dsk3:Miss Master:Y Slaves:NNNNNNN RasDial:Disconnected RasServe:Enabled
Disk failed or not present      

Internal or External disk. Internal disk is critical.

SA380 needs external disk when used as juridical logger.

 
Remote access heartbeat receivedRemote client connected to the logger and retrieved site name and firmware version.no parameters

Enabling RAS server             

Disabling RAS server            

Can be seen time-by-time in SA380 loggers to

make sure the RAS server is not hung up in call.

no parameters

The SA380TX-L logger has a few more messages worth including here.

SyslogDescriptionExample
Database startedPart of the normal startup. Version indicates data structure changes.Database started Database File Manager version 0.2
Slave configuredSlave processor got configuration. Optional parameters show available digital and analogue channels.Slave configured AnaMask=0x0003, DigMask=0x0000000F
Slave has started loggingStarted the slave processor logging cycle. It is not logging during configuration phase.no parameters

Errors

SyslogDescriptionExample
Thread stopped                  Software failure or possible overload. (1)thread=DiskLogger,state=8
Thread safety system restart    Software failure or possible overload. (2)no parameters
Unexpected disk removal         Error: only for Internal 
Battery level low at startup    

Flat battery; battery is not charging. Let the logger charge the battery for a few hours.

If the error still shows, send the logger back to MPEC for battery replacement.

2.9V

Event receiver overflow         System is overloaded. (3) 
Could not delete folder         

Happens during old file deletion. Usually indicates file system error.

Logger restart may fix it, if it happens because of clashing file access.

\usbdisk\www\days\20140817

Could not create file           Important for SA380: must have external disk installed. 
Modem command failed            

Serial port error; modem error. It is not an error during modem baud-rate detection!

Try to restart communication on the logger.

missing echo
M Tx File processing failed     

Slow comms (file has been removed since), file system error.

Logger had no connection to the server long enough to fill the message queue in the meantime.

Check in the syslog if the GPRS communication was lost for a long time.

Could not build:type:EBE tick:395137049463 seqno:0 count:1 file:\usbdisk\www\Days\20120910\ebierr\2012-09-10_MMM TX_0_0.txt

In case of system overload errors (1) (2) (3), it is a good idea to double-check the data collection settings. It is possible that the logger is collecting way too much data.

Acquire-on-change thresholds are the usual suspects. You may want to set the threshold higher.

GSM Modem

SA380TX

SyslogDescriptionExample
Modem power off                   
Modem power on                    
Modem command                   AT commands 
Modem command failed            check "Modem command" to find what triggered the error 
GSM signal strength at turn on    
Disconnecting RASdial           Ending GPRS connection 
RASdial disconnected            Important for SA380: must have external 
Creating RASdial entry          Have to add a number to Windows phone book to be able to call it with RasDial 
SIM registered to               Tells the network 
Could not reg. with GSM network International Mobile Subscriber Identity; International Mobile Station Equipment Identity 
GSM IMSI/IMEI                   International Mobile Subscriber Identity; International Mobile Station Equipment Identity 
Dialling RASdial entry          RasDial call; opening PPP connection; shows user,pass 
RASdial connected               RasDial succeded 
RASdial error                   err=Remote Access Error Codeerr=691
New RASserve timeout            RAS server started; timeout restarted 
DCD inactive (call cleared down)Modem data carrier line is down (connection lost) 
Disabling RAS server            RAS server stop 
DCD active (call connected)     TX2: Dialup has been called 
RING received                   Dialup during GPRS session 
GPRS started                    GPRS thread start 
GPRS ended                      GPRS thread end 
Modem fallback wait             Modem didn't manage to register to network, falling back for 1 hour 

SA380TX-L

SyslogDescriptionExample
Network registration failed  
Network operator  
AT command  
AT command failed  
Modem OFF  
Modem ON  
Signal strength  
PPP Connectshows user, pass 
PPP Connectedshows ip, gateway, dns servers 
PPP Disconnect  
PPP Disconnected  
Modem errorbaud rate not detected 
Modem fallback for  
Modem restartRestart comms 

MIMOSA push

SA380TX

SyslogDescriptionExample
Mimosa client started           Thread started 
M My IP                         IP change triggers MIMOSA disconnect-reconnect 
M Tx epRC    Request bootstrap    
DNS response                      
M Rx epRCr   Bootstrap          only response parsed 
Mimosa connected to II          epRequestControl has been answered OK 
M Tx sNCf    Send chan names    Important for SA380: must have external 
M Build Event                   

After 1.5 the logger only logs the fact that the message has been serialised.

tick:ticktime interval, smp:number of samples in message,evt=number of events

Events contain the samples.

time=2015-10-28T08:58:48.290Z,tick=1395864412451-1395864412500,smp=503,evt=50
M Tx sNDE    DAInt heartbeat      
M Tx sNDES   DADataSeq          

On old loggers (before 1.5)

ch:channel name, n:number of samples, time, tick, ch1:channel name, off1:MIMOSA offset, id1:calculated MIMOSA ID

ch:TC 1 Sec 4 TF n:7 time:2014-04-08T00:24:13.250Z tick:64424673636 ch1:TC 1 Sec 4 TF off1:577 id1:4750341636673

M Tx sNDES   DAWaveform         On old loggers (before 1.5)

ch:2325A N-R n:352 time:2014-04-07T03:12:25.070Z tick:55834692254 ch1:2325A WI off1:68 id1:4750350352452

M Rx HTTP 200-OK                On old loggers (before 1.5) 
No DNS response                 h:hostname, t:timeout

h:sntp.corp.ukrail.net t:36s

M Tx Couldn\'t send data        send() failed 
M Tx Socket opening failed      invalid handle or connect() error 
M Tx Send failed - no conn      No bearer; Too many retries 
M My IP changed!                Triggers MIMOSA reconnect 
M Tx No bearer                  No bearer; GPRS down, eth: no IP address 
M Rx No response from core sys    
M Rx HTTP error                 Logger stores the failed messages in mimosa/mimlog.txt503-Service Unavailable
M Tx Fallback for               retry in given time 
M Tx File processing failed       
M Rx Error processing           epRequestControl response parsing error 
Mimosa disconnected from II     Probably easiest to look for this and then trying to find the root cause. 
Mimosa client ended               

SA380TX-L

SyslogDescriptionExample
Mimosa client started  
Mimosa client ended  
M Tx Fallback forrandom fallback time 
M Tx Send failed - no conn  
M Tx Couldn\'t send databecause of HTTP 4xx status code; cannot send DataSeq etc. 
Mimosa connected to II  
Mimosa disconnected from II  
M Tx Couldn\'t send chan namesFailed to send 
M Tx Couldn\'t send heartbeat  
M My IP changed!  
M Tx sNDES   DADataSeqch:channel number in hex, off:MIMOSA offsetch:A800 off:2560 n:1 time:2014-11-10T15:50:34.495Z tick:1253
M Tx epRC    Request bootstrap  
M Rx epRCr   Bootstrapshows ntp, hosts, heartbeat 
M Tx sNDE    DABool digital  
M Rx epRCr   Bootstrap errordevice ID, missing parameter 
M Tx abortedIt's a complicated one, internal errors can trigger this! No available message bufer 

Syslog examples

SA380TX

DateTimeSyslog code
(for automatic processing)		MessageParametersDescription
15/09/201409:06:04234Driver information              PSUDriver.dll copied to \Windows\:1.0.0.1Normal startup installs operating system drivers.
15/09/201409:06:04234Driver information              Rs485Driver.dll copied to \Windows\:1.0.1.1
15/09/201409:06:04234Driver information              rs485isr.dll copied to \Windows\:1.0.0.1
15/09/201409:06:04234Driver information              SpeakerDriver.dll copied to \Windows\:1.0.0.1
15/09/201409:06:04234Driver information              TimerDriver.dll copied to \Windows\:1.0.0.1
15/09/201409:06:04234Driver information              watchdogDriver.dll copied to \Windows\:1.0.0.1
15/09/201409:06:04234Driver information              newSPIDriver.dll copied to \Windows\:1.0.0.1
15/09/201409:06:04234Driver information              oalioctl.dll copied to \Windows\:1.0.0.1
15/09/201409:06:04234Driver information              MpecSnmp.dll copied to \Windows\:1.0.3.1
15/09/201409:06:04125Config                          can't open \NandFlash\stw\CfgPlaceCfg.xmlNot an actual error message. The place of the configuration
is not set. Default: internal disk. It could be in CPU NAND flash.
15/09/201409:06:04234Driver information              Activated SNMP serviceOperating system driver has been (re)started.
15/09/201409:06:04234Driver information              Started SNMP serviceCustom agent driver has been started.
15/09/201409:06:0443Thread watchdog event           stop thread=EMPTY 
15/09/201409:06:041System startup                  1.5T441  4Sep14 serno=65532 Start of normal startup. Logger can write to disk.
15/09/201409:06:04125Config                          backup:Days\20140915\cfgchg\2014-09-15 09-06-12 MimosaCfg.xml 
15/09/201409:06:0479Config changed by MIMOSA        MimosaCfg.xmlticktime gets incremented which is part of MIMOSA
configuration, hence it updates.
15/09/201409:06:0443Thread watchdog event           start thread=Logger,tout=300000 mssyslog thread started. Thread watchdog has 5mins timeout.
15/09/201409:06:0443Thread watchdog event           start thread=Sysmon,tout=11000 ms 
15/09/201409:06:0443Thread watchdog event           start thread=miniQ,tout=30000 ms 
15/09/201409:06:0443Thread watchdog event           start thread=DiskLogger,tout=120000 ms 
15/09/201409:06:04234Driver information              Activated SNMP serviceRestarted SNMP service. Configuration has been set.
15/09/201409:06:04234Driver information              PROTOSLAVEDRV:RegisterData, RS485 owner=ebitrack, exitProtobuf slave module exits because RS485 belongs to EbiTrack.
15/09/201409:06:0443Thread watchdog event           start thread=EbiTrack,tout=90000 ms 
15/09/201409:06:0443Thread watchdog event           start thread=SCThread,tout=60000 ms 
15/09/201409:06:0443Thread watchdog event           start thread=Scheduler,tout=60000 ms 
15/09/201409:06:0443Thread watchdog event           start thread=mimosaClient,tout=14400000 ms 
15/09/201409:06:04130Mimosa client started             
15/09/201409:06:04188M Tx No bearer                   Conection profile set bearer (Ethernet or GPRS) is not active.
Mimosa data push falls back and tests it continuously.
15/09/201409:06:04128GPRS started                      
15/09/201409:06:0433Logger active                   ID=0 (ch 1 - ch 384)TX logger is active. (=Has valid samples.)
15/09/201409:06:0433Logger active                   ID=1 (ch 385 - ch 768)Slave 1 is active.
15/09/201409:06:0433Logger active                   ID=2 (ch 769 - ch 1152)Slave 2 is active.
15/09/201409:06:25100Modem command                   Auto-detect modem baudrate 
15/09/201409:06:2643Thread watchdog event           start thread=Netaccess,tout=300000 ms 
15/09/201409:06:27102Modem power off                   
15/09/201409:06:2849Daily status                    dsk1:366M dsk2:Miss dsk3:Miss Master:Y Slaves:NNNNNNN RasDial:Disconnected RasServe:Disabled 
15/09/201409:06:2852Available memory                40928k 
15/09/201409:06:2843Thread watchdog event           start thread=clearOldDayFiles,tout=3600000 ms 
15/09/201409:06:2843Thread watchdog event           stop thread=clearOldDayFiles 
15/09/201409:06:36132Mimosa server started             
15/09/201409:06:37103Modem power on                    
15/09/201409:06:52101Modem command failed            no responseBelongs to baud-rate auto-detect.
15/09/201409:07:50100Modem command                   baudrate=115200
15/09/201409:07:50100Modem command                   ATE1   
15/09/201409:07:53105Creating RASdial entry          ATD*99***1# 
15/09/201409:07:53100Modem command                   AT+CREG? [H+R] 
15/09/201409:07:57100Modem command                   AT+CREG? (roaming)Checks registration. Roaming is enabled.
15/09/201409:07:57100Modem command                   AT+CSQ   
15/09/201409:07:57104GSM signal strength at turn on  42% -90dBmGSM modem receive signal strength. It could be improved.
15/09/201409:07:59112GSM IMSI/IMEI                   IMEI:[353943040151148] IMSI:[234330421932261] 
15/09/201409:08:00100Modem command                   AT+CPIN?   
15/09/201409:08:00100Modem command                   AT+COPS=3,0   
15/09/201409:08:01100Modem command                   AT+COPS?   
15/09/201409:08:0291SIM registered to               Orange T-Mobile 
15/09/201409:08:03100Modem command                   AT+CGDCONT=1,"IP","ii-dte.corp.ukrail.net"   
15/09/201409:08:05106Dialling RASdial entry          usr=bypass,pass=password 
15/09/201409:08:26110RASdial error                   err=691Access denied because the user name, password, or both
 is not valid on the domain.
     We would need O2 SIM card to be able to connect.
The APN has to be enabled as well.
This can be an intermittent problem if the SIM configuration
has changed.

NTP Time synchronisation

DateTimeSyslog code
(for automatic processing)		MessageParametersDescription
10/11/201410:34:3343Thread watchdog event           start thread=SNTP,tout=300000 ms 
10/11/201410:34:33148DNS response                    h=ntp.exnet.com,ip1=79.135.97.70,t=0sNTP hostname is given by MIMOSA server epRequestControl message.
10/11/201410:34:33152Sending SNTP request            ip=79.135.97.70 
10/11/201410:34:34153SNTP response                   rtt=800,clockofs=-1203Round-Trip-Time must be below 2sec.
10/11/201410:34:3463Setting time                    to:2014-11-10T10:34:33.177Z by:SNTPSets clock since offset is greater than 1sec.
10/11/201410:34:3364Set time                        from:2014-11-10T10:34:34.390Z 

SA380TX-L

Time (GMT)CodeTextDetailsDescription
00:02.91System startup Normal startup. First message.
00:02.926Database startedDatabase File Manager version 0.2 
00:02.925Version informationMaster build = 725 
00:02.925Version informationMaster built on Jul  4 2014  13:08:45 
00:02.925Version informationSlave build = 725 
00:10.7103Modem ON  
00:14.0116Modem restart Restart Modem thread.
00:14.0102Modem OFF  
00:14.039Slave configuredAnaMask=0x0003, DigMask=0x0000000F2 analogue channels, 4 digital channels
00:14.039Slave configured  
00:17.133Slave has started logging Slave CPU is sampling and sending samples to master CPU.
00:24.0103Modem ON  
00:37.8104Signal strength+CSQ: 22,99

+CSQ:<received signal strength>, <bit error rate>

<rssi>: 0=-113dBm or less, 1=-111dBm, 2..30=-109..-53dBm, 31=-51dBm or greater

Shows raw signal strength and Bit Error Rate (99 = not known or not detectable)

00:37.891Network operatorOrange T-Mobile 
00:37.8100AT commandAT+CGDCONT=1,"IP","stream.co.uk" 
00:37.9106PPP Connectuser:default pass:MPEC 
00:48.1107PPP Connectedip=172.30.92.57, gw=192.168.111.111, pDNS=172.31.4.129, sDNS=172.31.4.193 
00:48.1134Mimosa alternate server started Mimosa server started on TCP 8080.
00:54.0130Comms push client started  
00:55.1161M Tx epRC    Request bootstrap Logger sends epRequestControl to server. (1st message to send)
00:57.8162M Rx epRCr   Bootstrapntp:ntp.exnet.com h1:raildaq.com h2:hostName2 hb:300Response from server. These are the exctracted parameters.
00:58.3171M Rx HTTP 200-OK  
00:58.3171M Rx HTTP 200-OKMimosa watchdog is armedLogger restarts in 3 days time if cannot transmit any more messages successfully.
00:58.3208Comms push client connected Successful epRequestControl message transmitted so logger is connected to server.
00:59.0163M Tx sNCf    Send chan namesunit configDefault user tags sent.
00:59.0163M Tx sNCf    Send chan names Actual channel names are being sent.
01:06.4171M Rx HTTP 200-OK Successful.
01:06.4132Comms server started Mimosa server started on TCP 80.
02:59.2152Sending SNTP requestip=79.135.97.70:123 
02:59.5157SNTP timeoutsleep = 60079s 
02:59.5153SNTP responseclock okLogger's clock is within tolerance. (1sec)
06:07.6168M Tx sNDE    DAInt heartbeat  
06:09.3171M Rx HTTP 200-OK  

Related articles