Slawomir Rzeznicki has written a BOINC wrapper for the project, which has been running successfully for quite some time now. This means that you can also participate by using an existing BOINC client or by installing one.
http://www.enigmaathome.net/The standard clients are going to be supported as usual, so nothing changes for current installs.
I am pleased to announce that the Scharnhorst message was broken in less that 24 hours. You can read the raw server logs as well as an interpretation and translation.
We are now dealing with first message again. This time we try our luck with the middle part.
We are taking a short break from the last unbroken M4 message in order to focus on another unbroken message that has been released recently:
http://cryptocellar.org/bgac/Scharnhorst.html
This message was probably enciphered with the M3 model.
As you may recall, we have been attacking the first and second parts of the unbroken message separately. The main server dealt with the second part, the old server with the first part. Since the old server had just been left in place to accommodate machines running the old software, the first part has gotten considerably less attention than the second part. The approximate numbers are 10 passes for each work unit vs. 60 passes for each work unit.
In order to fix this imbalance, the roles are now reversed. The work unit overview on the M4 Project page shows the statistics for the main server dealing with the first part of the message.
This run was started on March 28th 2006. |
The terminology has been an ongoing source of confusion. I hope that this analogy helps to clarify things:
Suppose that you have lost a gold coin in an area of one square kilometer. Fortunately, you have a metal detector. Unfortunately, the detector is such that you only have a 50% chance of getting an alert even if you hold it right above the coin.
You come up with a plan. Since you are a methodical person, you divide the square kilometer into 1000000 work units of one square meter each. Subsequently, you go over the whole area, processing each unit once. In 50% of the cases, you'll have found the coin. If you haven't, you'll go over the area a second time, unit by unit. Now your odds of having found the coin are 75%. And so on.
It should be clear by now that the square kilometer stands for the whole search space, which is divided up into 11388 work units. In the above statistic, we have searched each work unit about 60 times.
It should also be clear that the large number of remaining work units results from the fact that a maximum of 500 passes has been planned per work unit and the server reports the total number of units as 11388 * 500 = 5694000.
Willem Alexander Hajenius has written a graphical statistics tool which shows lots of information about the current work unit. Source code (GPL) as well as a Windows executable is available here:
http://hajenius.ifastnet.com/software/EnigmaMonitor/EnigmaMonitor.htm
If you want to compile yourself, you need a working wxWidgets installation.
On the unbroken message, all C-Thin/Gamma combinations have been tested at least once. Since it is unlikely that UKW C-Thin and Greek wheel Gamma were present on the submarines in November 1942, a single test of those combinations will have to suffice.
Today we resume the run on the second half of the split message that had started on March 28th and was interrupted by the C-Thin/Gamma run on June 22nd.
New versions of the client with the capability to connect to port 443 have been released. This should make it easier to run the client if you are behind a firewall and you cannot connect to port 65521. The new versions also introduce an installer.
For users who are already using version 0.75 there is no need to use the new versions.
Historians agree that UKW C-Thin and Greek wheel Gamma came into service in July 1943. However, there is a slight possibility that the wheels were already present on some submarines in November 1942 and had been used by accident. In order to check whether this is the case, the (new) server now hands out work units that include C-Thin/Gamma combinations.
The run is done on the full ciphertext. This means that it will take more time to process a work unit.
Since March 28th the project has been running two servers, each of them working on one half of the unbroken message. For the first half, each work unit has been revisited at least 7 times, for the second half the number is 16. As you may recall, applying the hill climbing algorithm several times with different random settings increases the chance of breaking a message.
It has been reported that the password expiry problem also exists in Windows 2000. Here's how to fix it:
Win 2000: change password expiry
In Windows XP Pro, passwords expire by default in 42 days. This means that the task scheduler will fail to start the client once the password has expired. You can disable this behaviour:
Win XP Pro: change password expiry
If the password does not expire, it is all the more important to choose a strong random password. If you haven't done so already, visit:
change password and background task
Some words about the state of affairs: A large number of people haven't upgraded the software yet. As a consequence we'll have two servers running for the foreseeable future. As everyone is probably aware, each server hands out one half of the unbroken message. On average, messages of this length need more restarts (equivalent to "walks through the search space") than long messages.
The old server splits the complete M4 search space into 75426 work units, the new server into 11388. Right now we have gone through the search space for 4.0 times on the old server and 4.7 times on the new server.
Since an element of chance is involved in the random restart hill climbing algorithm, it is impossible to say exactly how many restarts are needed until a break is achieved. However, statistical tests on the messages that have already been broken may be an indication on what to expect. More details are here:
New versions of the client software have been released. Please upgrade as soon as possible. The new versions have the ability to download new dictionaries automatically and have better logging enabled by default. The new versions connect to a second server using port 65521. The old server will still accommodate the clients that haven't upgraded. As a consequence there will be separate results/stats for the new server:
New Server ResultsHere's how to switch versions:
Unix Client UpgradeA note about work unit sizes: Due to changes in the software and the way the new work unit list is generated, the new work units come in several different sizes. Don't be alarmed if one work unit takes longer to process than another one.
There is a very real possibility that the first message did not break because there are missing letters somewhere in the signal. To deal with that, the signal will be split in two parts that are attacked separately.
The new server is now handing out the second half of the message. The old server will be handing out the first part of the message, so that the old clients' resources are not wasted. Still I would like everyone to upgrade, as the new versions are more efficient.
Today the second break has occurred. The signal in question is the third of the intercepts in Erskine's letter. You can read the raw server logs as well as an interpretation and translation.
We are now dealing with first message again.
We are now going for the third message. The first message has not yielded.
Amount of work spent on the first message:
Army model: all rings, 5 walks through the search space.
M3: all rings, 5 walks through the search space.
M4: middle ring at A, 10 walks through the search space.
This does not mean that this particular message can't be broken with this method. Some messages require many more walks through the search space before a break occurs. Also, we did not try all ring settings for M4 yet.
Kirk Pearson has created the M4 Project Wiki, FAQ. You can use it for reference or add experiences that might interest others.
For support questions that are not in the FAQ, please consider using the M4 Project Mailing List.
After the Slashdot article participation levels have risen dramatically. In the last 12h about 2500 unique IPs have connected to the server. In the last two hours 2310 work units have been submitted.
There are now two external stats pages, both of which use the primary stats source M4 Project Statistics:
Stuart Whelan has set up http://www.somepointinthefuture.com/m4_stats.html.
Alexander Haneng came up with http://www.hanengcharts.com/enigma/.
We have walked through the M4 search space 5.5 times now, no break yet.
I'll soon switch to check if the message was encrypted with M3/Army model compatible settings after all. For more information on how the server subdivides the complete M4 search space have a look at the entry on January 9th.
The Windows XP* default install instructions create user "enigma-client" with password "nominal", unless you've changed the script.
Obviously this is not good practice. The latest install instructions tell the user to replace "nominal" with a strong random password.
If you are already running the client, you do not have to reinstall. Just follow the change password and background task instructions.
Reported by Jamyn.
The work unit rate is now around 2650 per day! We are now through about 75% of the M4 search space (single walk).
The work unit rate is at a new record of 530 per day! The statistics will take some time to reflect that (see comments on February 19th). We are now through 15% of the M4 search space (single walk).
I've tried to break the other two messages under the assumption that they were enciphered with the same day key, so far without success. It looks like a full run on the other messages is required, so we are now attacking the first of the three messages. To avoid ambiguity, the message broken is the second of the three.
The first message has been broken. You can see the raw server logs here.
Thanks a lot for participating! I'd say we continue with one of the other messages in case they weren't enciphered with the same day key. More to follow later.
It appears that over the weekend the work unit rate increased to about 200 per day. This will take some time to be reflected in the stats, since the daily rate is calculated over the whole running time. If we keep this rate, a single walk through the search space could be completed in 30 days.
Note that the test messages referred to on the main page needed between one and six walks through the search space to be broken. There is always a little luck involved with hill climbing algorithms.
I released a new Windows version. It fixes a priority issue (reported by Jeff Gilchrist) and a return value bug in enigma-client.py.
Both fixes are not critical for the correct functioning of the software, so if you are happy the way it runs you don't have to reinstall.
However, the new versions now run at real idle priority, and the return value bug is plain ugly.
There is a side effect of running at base priority 1:
If you have other demanding programs running and you start the client manually (see INSTALL.txt), the client is pretty unresponsive, so you have to be patient when pressing Ctrl-C and such.
I have now included an enigma.exe in the source version. This means that people who have Python AND Mark Hammond's win32all package on their machine can now download the much smaller source package and follow the INSTALL.txt without compiling enigma.exe.
I don't know if the web pages make it clear, but the large sizes of the binary versions result from the fact that a lot of python libraries have to be distributed with enigma-client.exe, which is created by py2exe.
There was a network outage (over half an hour) at my provider. The server was running, but could not be reached.
We've walked through the second search space (M3 without Service, see below) for 3 times. As of today, keys from the third search space are tested. These keys can only be used with the M4 model.
Just a short status update: Results are now submitted from 10 IPs. In case you are wondering if things work as they should, from this end all looks fine. Clients are submitting in plausible intervals and the results are processed as expected.
If you have any questions, don't hesitate to write to my address at the bottom of the page.
After mentioning the project on the lfs-chat mailing list, results are submitted regularly from 4 new IPs (thanks, guys!). Now it kind of becomes a real distributed project.
Detailed install instructions are available online.
New Unix and Windows versions contain several bugfixes. Most of them deal with saving the results in a robust way.
The project has started! The server now hands out the second of the three messages. Since the Enigma M4 model is backward compatible with the M3 and the Service models, the server divides up the M4 search space into three parts:
Service Model search space
M3 (minus Service) search space
M4 (minus M3, minus Service) search space
First we walk through the second search space.
The software package now includes 00trigr.naval and 00bigr.naval. To create the statistics, I used 15K of original U-Boat signals. Unfortunately the signals had been cleaned up, so I attempted to recreate the raw form by applying the rules described in The Bletchley Park translated Enigma Instruction Manual. I can assure you that this is boring work! Here comes an example:
Cleaned up version: Mittags Qu. 85862 durch Feuerüberfall Nachschubkutter etwa 50 t schwer beschädigt. Leichte Wabos.
My attempt at a raw version: mittags qu x act funf act secs zwo durc feueruberfall nacscubkutter etwa funf nul tonnen scwer bescadigt x leicte wabos
Especially the use of C instead of CH makes the text look slightly ridiculous. I wonder if that rule was always observed.
Some words about the the break of challenge message 5:
At the time the server only printed new high scores to the logs. This had worked well for the other challenge messages. Break 5 though would have been lost due to the higher score that had been submitted on Dec 5th.
Luckily I restarted the server on Dec. 15th, which wiped out the reference high score that the server keeps in memory.
After this experience I decided to print every score that is at least 80% of the current high score. This makes the logs less pleasant to read, but future partial breaks should be safe.