WASD update additions
The update left two links in error, both concerning performance logging: HyperSpixx, showing stored performance data, and Mondesi, for real-time control. The first one requires an agent that stores data in files, and these are presented uing a CGI-based execuatble. That program can no longer determine the nodename for which data is to be displayed, and when I restarted the agent, it failed without notice…
The second program is started from the browser, and requests data from the system using the system API to collect performance data, and displayes it in the browser window. The link does display the window but slightly distorted, but no centent….
Today, I installed the latest versions of these programs.
Mondesi – now version 2.1 – was installed without a problem, and it now shows the correct data.
HyperSpi – the WASD version – has been buiklt and installed, the agent started; it does start and created a file. But displaying the data still fails: No node supplied….
The version by Jean-François Piéronne however causes problems in building. It comes as a complete package that must now be compiled and linked – where originally, the object files were supplied and LINK was sufficient. so you now need a C-compiler, and, for the libraries used, you must use MMS (or Hunter Goatley’s MMK), if you want to use the BUILD.COM procedure.
I found the DESCRIP.MMS files used in buidling the library are incomplete – so I had to update the files before I could create the agent and web-application. The agent has been installed and works – but the web-application still gives an error ‘No node supplied’ – and the documentation that is supplied doesn’t show a reason….Or how to define the node. But here, there has been no change….
So still some work to do…
The terminal amulation over port 80 – using WebSockets – works, once a [world] entry was added to WASD_CONFIG_AUTH.CONF:
if (websocket:) * r+w,https:
before the authentication for the site. And this addition prevents direct access to the terminal connection., since the site itself is still acessed over SSL, and requires login just for that. In the admin page however, you won’t see the user that started the application. But that could well be handled by the logger.
It took some preparation because quite a lot of basic config of the server (not the sites) has changed and so the process needed some time: Another naming convention and location of logicals, and a change in configuration-schema made this update less straight forward than normally is the case. Including full startup (and shutdown) of the web server – and surrounding software: like the PHP engine and mail support.
Not forgetting the daily, weekly and monthly processing – it all needed to be overhauled.
I have done a bit more investigation on the PHP issue.
First, I created a procedure in which I can set the PHP environment as /JOB logicals so I can do the tests without interferendce with the access over the Internet.
Next, I checked the WordPress versions I haev installed. Apart from 2.6.3, I have installed 2.6.5, 2.8.2 and 3.4.1. I tested each of them after setting the PHP version to use (either 5.2.6 of 5.3.14), directly on the root directory of that WP version – so VERY basic.
Even using 5.2.6 I had problems – where this wasn’t the case a few days ago: On each of the blogs, PHP complained a file could not be located:
$ set def WP : ! the root directory of that WP version
$ php index.php
Work at hand
Apart from the PHP issues, there are a few other things under construction: A new homepage, and a suite to process network-related logfiles.
For the new homepage I plan to use Mark Daniel’s VmsWasdContentManagementSystem – a native VMS executable that can handle this type of posts – even blogging is an option (perhaps, any blog on this site may be redesigned using this package). I had the beta installed, so I removed it to prevent problems that coud arise; downloaded the latest version, (both the sources and the AXP objects), built and installed it. It does require some configuration, and mapping in WASD, and to get famliar with it (and because of the recommendation) I set up the example as in the documentation. But either I don’t understand or mis-interpret the docs, or these are inconclusive (incomplete of plein wrong – I cannot tell), I ended up with a message:
ERROR 403 - reported by VWcms
Site directory not configured!
To be investigated….
It’s an idea for quite some time: Scan all incoming network access, find out who’s attempting to hack, or abuse the systems, and shut the door for these people.
I started today with a program to scan the SYSLOGD logfiles on Diana: the firewall on the dge of the domain logs all access in this file, and when it is over 25.000 blocks in size, it’s cycled, and all cyccled files are stored in a zip file during the monthly maintenance process. Other files to process are the PMAS and FTP logfiles, and the access logs of the webserver.
So I need a program to convert these files into data that can be stored and analyzed, and that is also capable of updating the firewall with the top-100 addresses; the Vigor is capable of storing 192 single addresses, address ranges or networks that can be denied access – at the gate.
I started with a DCL-procedure that splits the SYSLOGD output – either active or archived – into incoming and outgoing traffic; each of which is next split into protocol-specific files; so at that moment, I have all lines of logging for every protocol, either incoming or outgoing – in exactly the same, fixed format. Therfore, it’s very easy to extract the required data from these files: date and time of access, the source and destination address and port – and the protocol.
Since there is quite a number of archives to process, I also created a procedure to scan a directory for these files – put there by hand of by unzipping an archive – and have each file processed that way. I’ve taken a decision to mark each final output file by the date it is created, and once created (if not existing) it will be extended with each SYSLOGD file that is processed.
This works fine now – next is the extraction of the same data from the PMAS logfiles, but IIRC, that has been done already, I just have to look fro them; otherwise, it is not a lot of work to do the same for these files. The same applies to the web-server access logfiles: Create a procedure that can handle one, and I’m done (just add a wrapper that passes the filename of the file to be processed.).
And, of course, a program to store this data into a database, a program to analyze the data, and one to update the firewall accoringly.
A few days ago, I found out – by accident – that the PMAS license expires tomorrow. I sent a request for a new license to the address I know exsists for that type of message – but it bounced. Next, I sent it to the address of Hunter Goatley – who’s in charge of the hobbyist licenses – and that bounced as well. So I sent it to the support desk of Process Software, but since I have a free license, they couldn’t help me; in stead they passed another address – which bounced also, so I was advised to contact Hunter directly – which didn’t bounce for the next hour. So it is likely to arrive; hopefully Hunter is not on holiday, and the license arrives is time – or I’ll be buried under all the messages that PMAS is now blocking ro rejecting…Fingers crossed….
PHP upgrade (continued)
Mark found something – because Variable-format files are indeed a problem. This type wasn’t taken into account in his rigorous testing…
Now he has found what was going wrong and has released a new PHPSHR file – AND he’s now able to reproduce the error. For now, all files that are included (’require’ or ‘include’ in the code) MUST be stream_lf. So on request, I converted the only file that is in variable format, to stream_lf. PHP 5.2.6 has no problem – but when I tried with PHP 5.3, PHPWASD.EXE stopped without error – and returned a 500 error.
Running the blog with PHP.EXE didn’t change a thing….
PHP upgrade – continued
Got a reply from Mark – I sent him some WATCH output on the errors I encountered last Friday – and he noticed PHP code in the log – where it shouldn’t.
Admitted: I completely overlooked this part of the output….But these lines explained both the database-access problems with WP 2.6.3, and the shown PHP-code using WP 3.4.: the log shows no space, or better: line termination, between the PHP starttag and the first characters of the next line:
where you would expect
like the files read.
I encountered the first of these when I ran the blog using WP 2.6.3, with PHP 5.3. I thought the absence of the mysql and mysqli extensions were to blame, but that seems not to be the case, but the fact this file wasn’t interpreted. Apart from the other messages (that now seem to be suppressed in the PHP.INI file) this was unexpected, but it explaines the database access issue: the file is not included and interpreted but simply outputted – since there is no starttag stating this is PHP-code: the system simply concatenated the tag (which does exist in the file) and the next line – causing “
<?phpdefine“. This is the databse-access file, kept outrside the normal path, but included in wp-config.php. And, in the WP 2.6.3 environment, it is the only file in Variable format.
As for the second example, this is a plain WP 3.4 file – the one I have converted to Variable format on request of Mark. Again: the lines are simply concatenated – WITHOUT addein Linefeed or NULL character, causing it to be interpreted as plain text to be outputted, in stead of being PHP code to be interpreted. Therefore, the database access file is never reached, so you would never see the first message showing up.
More on PHP update
Downloaded the latest PHP (5.3.14), in which all extensions are now included, but it has it’s own obstacles: WordPress 2.6.3 can no longer connect to the database; I ran into something similar when I tried to bypass the mysql extension in favour of the more modern myslqi one, where WP signaled it missed an extension…No message of that kind in this case, but a database connection cannot be established…
Not such a big deal – WP 3.4.1 is on the shelves to be rolled in – when PHP 5.3 works.
So I continued testing.
Not that it mattered much: I got the very same issue: missing end-tags at EOF, on the every first file that is ‘required’ in the code on index.php:
* Front to the WordPress application. This file doesn't do anything, but loads
* wp-blog-header.php which does and tells WordPress to load the theme.
* @package WordPress
PHP update tested
Last mail I got from Mark Berryman stated that PHP now uses mmap in memory management – and that rang a bell: the wget image, ported by Steven Schweda, does as well, and that one crashed on ACCVIO in accessing robots.txt, according his investigation. He suggested there might be some issue with the MMAP implementation in ACTRL – the C run-time Library. If that’s the case, it may explain why PHP 5.3 has trouble as well.
Last OS update included an update of this library, so perhaps it was solved.
But no: again, the PHP engine failed to recognize EOF as the missing “?>” end-tag. So that’s to be reported once more. It must be something withing the environment since, as he told me, he couldn’t reproduce the error.
At about 22:00, I couldn’t get to the webmail agent – got a “503″ error. Checked the server and found about 20 helper processes, many of them running PHPWASD (so accessing PHP code) and in LEF state. But the administration pages of WASD showed just one or two.
I tried restarting the server but that didn’t help.
I tried STOP/ID for each of them, no error but the processes didn’t go away.
The only way to get around it was to restart the machine. Next, I changed the throttle, it allowed for 20 processes and even more. So I lowered the number of processes for all blogs.
My (about 5 years old Pentium-4 HT) workstation dubbed Aphrodite will get another role: It will replace the machine in the living room. That one is Pentium-4 as well – without HT, and quite problematic at times.
I obtained a state-of-the-art new box: ASUS P9X79 motherboards with Intel i7 processor @ 3.6Mhz, and 8 Gb of memory (expandable to 64Gb). 8 channel audio, as on my previous system. Transferred Video and disks, but not the DVD-drives: These are ATAPI and the new system has (e)SATA only, to I had to obtain that as well.
Suitable for the heavy stuff I intend to run on the beast: Running multiple Alpha emulators side by side, and processing sound, image and, perhaps, video. I could use Linux on the box, but the emulators I can use do either not run on Linux, or not in a way that I intend to use them. For sound- and image processing, I already have Windows-based software I can work with pretty well, and I would need to learn these Linux equivalents as well. In the future I may add Linux as an alternate OS but for the moment, I stick to Win7pro-64.
Some trouble: The front USB bus has a different connector that doesn’t fit anywhere on the motherboard ans yes: I do need them; there is no COM exit, it needs to be added, and the motherboard seems to have a broken DIMM slot so 4 Gb of memory isn’t fitted in it’s preferred position.
These I’ll have to address with the supplier.
But I had to install the OS from scratch since Win7Pro-32 didn’t boot on this box. But installing the OS didn’t work out as good as I expected: I had to clear the whole disk – including the partitions containing data – because the BIOS of the new box couldn’t handle them: this is EFI based….
It was no problem to move them to the other disk – I thought – using a DOS box and XCOPY the contents to a directory on the other disk. But once that was done, I couldn’t find the directories I created. No big deal. Pity – but I do have a backup, and there hasn’t been much changes after that anyway. Of would it be a disk I didn’t expect?
After shutdown and moving the machine to its fibal locatioen, it turned out that a boot after shutdown did almost always fail, and I had to do a repair from the installation disk – which invariably failed because “… the system to be repaired is incompatible…” . But when offered to reboot normally, there seemed to be nothing wrong. Might been caused because I installed in safe mode?
So in the end, I re-initiated the disk, and installed everything again from scratch – but now when booting from the DVD-drive. From that moment on, it all went smoothly. Getting drives and software from in Internet – no problem.
But in accessing any of my own sites, there was. None responded, but services and servers were up and running….
New IP address
First thing to be done is pinging the server by name:
translated the server to be 22.214.171.124 – what I would expect because that’s what’s in DNS, being the outside address of the router.
But when accessing the router to seee what it says, the WAN connection is now on a different network, as well as the DNS-resolvers of the router. Contacted the ISP site (luckily there was no problem getting out!) and found there had been done some work that morning, and the connection had been down for a few minutes. So I called the help desk, and it was confirmed that the address had indeed changed. but there had been no information on this – which I would expect to be sent IN ADVANCE. Anyway, I had to contact the registrar of my domain to have the DNS references updated. That requires a signed document, which could be sent as an attachment in an email message.
Which I couldn’t use over the Internet….
But there are other addresses I could use: my provider’s, gmail, yahoo, hotmaill….So I created the letter, signed it, scanned it (using the new box – even with it’s problems) into a PDF file and mailed it. Next by the phone, it was handled within minutes, but it took some time before it would be expanded over the Internet.
This morning however, it still didn’t work: Although the new DNS-servers got them (the router configuration shows their addresses) the DNS servers inside the LAN didn’t. So I restarted BIND on the VMS box, and the router, but in some way or another it didn;t help. Looking into the LAN configuration of the router, I found the DNS-server in the LAN was the VMS box…Removed that: and now it’s all working again. But from elsewhere – mail, in particular – it may take some time: I didn’t mention that so that still refers to the old address..Will be changed today as well.
There has been one advantage: No spam either
Clean-up of DHCP and local DNS
Over the years,. systems have come and gone, and any new node in the LAN get’s an address by the DHCP server; and as long as the MAC address doesn’t change, that address will get the once supplied address. That will add these systems in the local DNS – and they’ll stay there.
But systems come and go, and the obsolete references are not deleted. So I took the possibility to remove all the old entries in both the DHCP and BIND databases.