I've been curious for a while about custom map files and what's behind the binary format. I poked around some, and it turns out that there is a (relatively) simple explanation!
Note: this info is based on the latest Creeper World (v0572) and Creeper World Map Editor (v0380), running on Windows.Quick answer: map files are XML data files compressed using the zlib (http://en.wikipedia.org/wiki/Zlib) / deflate (http://en.wikipedia.org/wiki/DEFLATE) algorithm, the same compression algorithm used in ZIP files.
Long Explanation:To convert a .cwm file into a .xml file, you need a program that can compress/uncompress using zlib (it doesn't work to simply treat it as a .zip file). You'll likely also need to know how to use the command line.
One tool set that can do the extraction are the offzip (http://aluigi.org/mytoolz/offzip.zip) / packzip (http://aluigi.org/mytoolz/packzip.zip) programs (if you use these, make sure you have the latest version of them).
- From the command line, offzip.exe my_map.cwm my_map.xml 0 will create my_map.xml by uncompressing my_map.cwm
- After that, you can edit the XML file by hand (see below for details).
- Finally you can use packzip.exe -c my_map.xml my_map.cwm to compress the XML file back into the custom map format.
- One additional concern: packzip v0.2 might complain about missing libgcc_s_dw2-1.dll; you can download it here (http://www.down-dll.com/index.php?file-download=libgcc_s_dw2-1.dll&dsc=) and put it in the same directory as packzip.
Or, if you know a programming language with the zlib/deflate support, it isn't that hard to make a program to do this for you. (see below for mine)
Technical details on cwm format:
- The compressed zlib format is also known as RFC1950 (http://tools.ietf.org/html/rfc1950).
- The first two bytes in the file are 0x78 0xda, which is the zlib header meaning "deflate algorithm, 32k window, max compression"
- The next bytes are the XML file compressed using deflate
- The final 4 bytes are the Adler32 check sum of the input file
A couple of comments on the XML map format:
- Units of time are given in frames (36 frames per second)
- The grid runs from 0 to 47 in the vertical (y) direction and from 0 to 69 in the horizontal (x) direction.
(0,0) is in the top-left, (69,47) is in the bottom-right (you can see this in the map editor). - Terrain and wall data are stored as a list on numbers running from left to right and top to bottom: (0,0), (1,0), (2,0) ... (69,0), (0,1), (1,1) ...
The values are 1-5 for terrain (giving the elevation at that point) and 0, 1, or 1000000 for no wall, normal wall, indestructible (crazonium) wall - Most of the other game objects have parameters gameSpaceX and gameSpaceY for their coordinates
- Totems are listed under places
- Odin City and the Rift are under specialplaces, and have seemingly-unused health and energyLevel parameters
- Odin City is called BaseGun
- Artifacts are called Ruins
- The text for title, opening, and artifacts is html-escaped (space is %20, etc)
- Custom background (if any) is base64-encoded image data (the game and editor are ok with .jpg or .png data of any size and quality initially; if you use the editor it converts any custom background to a 700x480 jpg @ 85% quality). An online base64-encoder is here (http://www.opinionatedgeek.com/dotnet/tools/base64encode/).
- backgroundMap is 0 to 5, and must be given even if there is a custom background
- Colors are the decimal equivalents of the hexadecimal 0xRRGGBB
Attached are FormatDemo.cwm and FormatDemo.xml (as a .zip), a map with two of everything in place so you can easily see the syntax of the XML file.
Ways to use this knowledge about the custom map format:
- Easily delete all collectors/mines/totems/...
- Edit spore waves by hand
- Set emitter settings to precalculated values (such as stacked emitters whose intensity doubles each minute)
- Automatically generate a custom background based on the terrain, or vice versa.
- Create a map with randomly-generated terrain
- Someone very ambitious with too much extra time could even write their own map editor program, with whatever desired features (circle terrain brush, undo, etc) that the current map editor doesn't yet have.
However:Being able to view and edit the underlying XML data for the map is mainly a time-saving tool than anything else.
About the only thing I found I could do
only by editing the XML by hand was set the terrain to 0 or 6 (which the game still displays, giving two additional elevations to work with). Also, you can set walls to other values than 0, 1, or 1000000 (a 2-value wall survives about twice as long as a 1-value wall; the editor doesn't display it even though the game does), and you can tweak timing values in smaller increments than tenths of a second.
A word of caution: the map editor doesn't display any useful error if you give it a badly formatted map, and sometimes needs to be restarted if you really mess up its internal state.
For the curious, I have attached cwm_edit.pl (as a .zip), which is the program I wrote to help me automate this conversion process. It is a perl script to be run on the command line. Feel free to use/copy/adapt it however you want (if you can run it).
(I am also working on some terrain-generation and image-conversion programs, which I'll share if/when I'm satisfied with them)
Edit: I clarified some of the text and attached a couple screenshots of this technology in action.
Edit: added link to base64-encoder, added "tenths of a second" discussion.
Interesting, i'll have a look at working with this later then ;)
Oooh, could be used to create even more interesting maps, and also have walls that decay faster/slower. :O
Very professional analysis mopa! I read your post before, but I awaited Virgil's approval before responding.
I also decompressed some CWM files in the past (with Virgil's help) to be able to shift maps by a few blocks for instance (the terrain height array that is) by programming a tool for this, but you have found some very interesting parameters that I'm certainly gonna play with.
Thanks for your detailed post! :)
I asked this before but Virgil didnt answer.
http://knucklecracker.com/forums/index.php?topic=5131.0 (http://knucklecracker.com/forums/index.php?topic=5131.0)
I can't get packzip to work
Quote from: thepenguin on November 06, 2010, 07:55:38 PM
I can't get packzip to work
Can you give some details, such as:
1) What exact problem are you having?
2) Does the packzip program run at all?
3) What exact command are you using?
4) What version of the program are you running?
5) If you've tried using the corresponding offzip program, have you also had issues with that?
1)DLL is missing
2)yes
3)packzip.exe -c my_map.xml my_map.cwm
4)the one you linked to
5)yes, no
could you just packzip them for me?
I send you xml, you send cwm
Quote from: thepenguin on November 07, 2010, 08:13:43 AM
1)DLL is missing <- Download the DLL
2)yes <- good.
3)packzip.exe -c my_map.xml my_map.cwm <- congrats.
4)the one you linked to <-w00tyt00ty
5)yes, no <- mk.
Dll does not download
can one of you who has it working just pack it for me? (I'd like to know before I try editing xml)
Quote from: thepenguin on November 07, 2010, 01:42:30 PM
can one of you who has it working just pack it for me? (I'd like to know before I try editing xml)
Attached xmltest.cwm, and xmltest2, 3, 4, and 5 for thepenguin.
And now: packzip_offzip_and_dll.zip, which contains (surprise, surprise) packzip.exe, offzip.exe, and the DLL needed to make packzip work, all in one convenient location. (All are exactly the same as from the three (http://aluigi.org/mytoolz/offzip.zip) download (http://aluigi.org/mytoolz/packzip.zip) links (http://www.down-dll.com/index.php?file-download=libgcc_s_dw2-1.dll&dsc=) I gave in my original post, by the way).
One other thing: if anyone is looking for a good XML editor, I use the free Notepad++ (http://notepad-plus-plus.org/), which gives useful color-coding and collapsible structure.
Slow decaying walls!!
Quote from: thepenguin on November 07, 2010, 01:42:30 PM
Dll does not download
can one of you who has it working just pack it for me? (I'd like to know before I try editing xml)
Google and Download the DDL...
Agree on above post(and hi Kam :P)
Just used some of the above info to create a new map. I like the ability to use 2 additional elevation levels, but unfortunately I can't set their tint value. It seems like the level 0 tint is fixed at 0.2 or something close to that and the level 6 tint at about 1.6.
It would have been great when these 2 extra levels could be used in the map editor as well, including setting their tint.
Edit: adjusted level 6 tint value.
Quote from: mopa42 on October 29, 2010, 10:28:04 AM
Custom background (if any) is base64-encoded image data (the game and editor are ok with .jpg or .png data of any size initially; if you set the image using the editor it converts it to a 700x480 jpg @ 85% quality)
I really hope Virgil remembers what he said earlier about
custom background compression (http://knucklecracker.com/forums/index.php?topic=2780.msg18675#msg18675) (
hint hint ;)). I know there will be a final patch of the map editor (i.e. with Chronom type map generation), so I hope this will be improved as well, because the graphics I created after a lot of work currently get reduced a lot in quality....
Quote from: UpperKEES on November 14, 2010, 06:44:01 PM
the graphics I created after a lot of work currently get reduced a lot in quality....
I think I can help.
Here are some more details on custom backgrounds.
You can manually edit the files to avoid the 85% quality restriction on custom backgrounds. First base64-encode (http://www.opinionatedgeek.com/dotnet/tools/base64encode/) your image (in whatever jpeg quality you desire). Make sure that the data isn't broken up into multiple lines, then add it in the XML file like this:
<custombackground>/9j/4AAQSkZJRgABAQE....</custombackground>, and finally convert back to .cwm format. Alternately, my cwm_edit.pl program can do this step automatically.
The only caveat is that the map editor will reconvert the custom background to 85% quality whenever you save. So,
for a high-quality custom background image, set it manually as the last step in creating your game. (I haven't actually tested submitting a map using this method, but it certainly would work for maps on one's own computer).
Oh, great! That's very useful, thanks! :)
I won't be able to run the PERL-script, but won't have a problem doing it manually I guess.
Notepad and other loose tools ftw, eh? ;)
Quote from: mopa42 on December 12, 2010, 06:09:24 PM
First base64-encode (http://www.opinionatedgeek.com/dotnet/tools/base64encode/) your image (in whatever jpeg quality you desire). Make sure that the data isn't broken up into multiple lines, then add it in the XML file like this: <custombackground>/9j/4AAQSkZJRgABAQE....</custombackground>, and finally convert back to .cwm format.
I just tried it with a 1.3 MB PNG-file (zero compression). The online base64 encoder froze my browser, just like another online version I tried, but after installing a Firefox base64 encoder as Add-On it worked like a charm.
Quote from: Echo51 on December 14, 2010, 03:49:09 AM
Notepad and other loose tools ftw, eh? ;)
I actually tried Notepad, but it seems to have some problems with large file sizes, so I used WordPad instead. I replaced the custom image string with the new one and converted the XML file back to CWM format. After copying the map to my customgames folder it loaded without any problems and looks great! 8)
The resulting CWM-file is now 336 KB instead of 104 KB, so nothing to worry about.
Thanks mopa42! :)
Quote from: mopa42 on October 29, 2010, 10:28:04 AM
Ways to use this knowledge about the custom map format:
- Easily delete all collectors/mines/totems/...
- Edit spore waves by hand
- Set emitter settings to precalculated values (such as stacked emitters whose intensity doubles each minute)
- Automatically generate a custom background based on the terrain, or vice versa.
- Create a map with randomly-generated terrain
- Someone very ambitious with too much extra time could even write their own map editor program, with whatever desired features (circle terrain brush, undo, etc) that the current map editor doesn't yet have.
About the only things I found I could do only by editing the XML by hand:
- Set the terrain to 0 or 6 (which the game still displays, giving two additional elevations to work with).
- Set walls to other values than 0, 1, or 1000000 (a 2-value wall survives about twice as long as a 1-value wall; the editor doesn't display it even though the game does).
- Tweak timing values in smaller increments than tenths of a second.
- Bypass the 85% custom background compression.
Your latest addition about tweaking timing values can be very useful, especially when you like to use an emitter with a frequency of for instance 6 frames, so only blasters with the fire rate upgrade can synchronize with it, or multiples of 7 to allow blasters without the fire rate upgrade to synchronize.
A few more additions (only possible by hand):
- Lower or higher entire terrain levels by using find & replace.
- Shift the entire map a certain amount of blocks up, down, left or right, which can save you quite some time when the terrain is already finished. I wrote a little program to do this for me; just let me know if you need it.
- The use of intensities higher than 1000 for both spores and emitters.
- The use of more than 1 decimal for specifying intensities (only useful for very precise tweaking on for instance puzzle maps).
- The use of negative values for spore intensities which will result in 'soaking sponges'. (This does not work for emitters, because negative creeper doesn't spread.)
- The use of delays larger than 9999 seconds.
Happy map making! 8)
XML editing could be a solution to this (http://knucklecracker.com/forums/index.php?topic=1000.0)
I know this thread is old, but this is amazing...
Upperkees told me about it in chat and I believe i'll have to try this sometime :)
I have tried both programs (packzip and offzip) and have tried to use the Command prompt but the system is not taking it.
Is there a program that will work for windows?
You have to use offzip from the command line to decompress the CWM-file:
offzip.exe my_map.cwm my_map.xml 0
What version of Windows do you use and what is the error message you get?
Here's one of the images in the demo screens zip, showing the two commands in action. This might clarify things if you're still having problems.
I downloaded both the offzip and packzip, but they don't have those files.
And if I try to run either .exe, it just runs a line of code through the command prompt and exits before I can see anything. And it does nothing.
I am using windows 7 and I got the "'offzip.exe'" is not a recognized file" type error message from the command prompt.
Quote from: Kingo on February 06, 2013, 10:21:38 AM
I downloaded both the offzip and packzip, but they don't have those files.
Of course you have to replace the file names of the maps with the file names of
your maps.
Quote from: Kingo on February 06, 2013, 10:21:38 AM
And if I try to run either .exe, it just runs a line of code through the command prompt and exits before I can see anything. And it does nothing.
I am using windows 7 and I got the "'offzip.exe'" is not a recognized file" type error message from the command prompt.
That's because they are not included in the search path. I recommend you copy those files to the same folder as your CWM files.
Then use the CD (change directory) command at the command prompt to change folders before you execute offzip (or open the command prompt in that folder from the Windows Explorer).
If you don't know how to use CD, you should consider not to mess with your CWM-files anyway.... :P
I give up...
Hello mopa42 and all other who are reading this.
I am a Linux User who is not able to run the Editor on Linux nor on Windows, so i decided to take your hint and started to write my own multi platform Map Editor based on the excelent information's you shared in this post.
By now i am able to decode .cwm files into .xml files back and forth with a standalone application, no need to install anything.
I attached the Linux 64 Bit version, creating a Windows version is no problem (if wanted).
At the moment there is no exporter of the custom background images, even if i am still able to decode it. If someone is interested on the progress or the actual versions feel free to reply or contact me.