Car Monitor Ver. 1.2 (a.k.a. "carmon")

Back
Latest Update: April 7, 2004

What is it?

Carmon is designed to log where a car has been. Pretty simple idea huh?
It does this by recording the output of a GPS receiver (NMEA: 4800bps RS232) to a compact flash card working in
ATA emulation mode (looks like a hard drive to the PIC, in fact you could use this with an IDE hard drive).
The compact flash card is formatted in the FAT16 format and can be read by any computer capable of reading a FAT volume.

What does it consist of?

Carmon consists of 4 major components:
1. 16F877 20MHz PIC Microcontroller
2. 24LC256 I2C EEPROM
3. Compact Flash card - 32MB tested so far
4. Firmware for the PIC: Programmed in C using PICCLite from HiTech
Obviously other support components are required, including a variety of capactors, a 16MHz crystal, and a MAXIM rs232 level convertor chip. Please see schematic for components required.

A way to get a programmed PIC is also required. If you have or have access to a programmer a HEX file can be found below.
If on the other hand you have no access to a programmer a preprogrammed PIC can be purchased from me. Please click here for further details.

What does it do?

On powerup, Carmon makes a few self checks, i.e. is the card alive? is the EEPROM there?
This check gathers the number of heads, sectors and cylinders the card reports for use under emulation.
(Note that this project does NOT use LBA, for some reason the few CF cards I had simply would not function under LBA, therefore it was decided to go ahead using CHS addressing).

Once preliminary checks are complete some vital parameters of the FAT file system are required, followed by finding the first sector of the file is free to write to:
1. The partition table on the card is read to determine the first sector of the first partition.
2. The first sector of the first partition is read. This sector contains all the vital parameters of the FAT file system
3. Using the information gathered from the first sector the root directory is located. It's first sector is scanned for the storage file, in the current version it scans for the first file starting with CARMON. The file's starting cluster and total length in sectors is recorded
4. The first sector of the file is loaded. The first byte is checked, if it's value is 0x00 Carmon has found the sector to write to. If not it continues to scan sectors sequencially until it finds one where the first byte is 0x00. (A text file normally never has a 0x00 character, which is what Carmon uses to determine where it last wrote to the file).
With the first free sector of the file now found Carmon begins "listening" to the serial stream coming in.
It reads in bytes until it has enough to write one sector (all operations on a hard drive must be done on a sector by sector basis), at which point it writes what is has gathered to the drive (the CF card).

The purpose of the EEPROM is to act as a 512 byte buffer. Since the PIC being used does not have enough RAM (I know, I know, should have used an 18F part, didn't have programmer support for the 18F's when I started out, so there!) to store one sector, the EEPROM is used as storage.

To prevent premature failure of the EEPROM (from writing to the same locations over and over) each subsequent collection of a sector to write is distributed over the whole EEPROM.
Writes to the EEPROM are done in 64 byte chunks since the EEPROM requires 5ms to complete a write cycle (single byte or page, doesn't matter), while the input serial stream has a new character every ~2ms.

I have also created software to aid in the use of the project:
1. create_wiped_file.c - Creates a file full of zeros of supplied length.
  Use this program to create a file to place on the CF card.
2. extract_text.c - Extracts all text up to the first 0x00 character to specified file.
  Use this program to extract the data written by carmon to the CF card.
3. proc_gps.pl - Perl script that seperates a GPS log file into seperate "trips".
  Use this program to split up a stored log of GPS output into seperate files for each trip, each trip being sorted in an appropriate directory based on date.
4. GPS_client - Outputs the contents of a GPS log file out the serial port, displaying relavent details.
  Used in conjunction with a copy of GPS mapping software and a null modem cable to "play back" a trip.
  By loading a log file you can follow the trip on the display of any compatible GPS mapping software.
  I use Microsoft Streets and Trips 2002, but it should work with any GPS program since the output looks like it comes from a GPS unit.

Limitations?

Since the PIC is a very resource limited platform, and the FAT file system was designed for a platform with far more memory on hand, Carmon has a few limitations:
1. Carmon does not actually modify or even READ the FAT. It simple writes to a file that's already on the drive. Therefore the file selected MUST be contiguous.
2. The partition the file is on must be the FIRST partition.
3. The file's entry in the root directory must be within the first sector, therefore it must be within the first 16 entries.
4. The file's name MUST fit the old DOS 8.3 style of file names, therefore carmon.log is valid, car_monitor.log would not be found.
5. If the file does not end on a sector boundary the contents of the file in the last sector, BEYOND the length specified will not be read in by your OS since it relies on the file length given.

Requirements?

Requirements to ensure proper operation are:
1. I'll say it again: the partition the file is on MUST be the first partition.
2. The drive must be formatted FAR 16. While FAT12 MAY work (I've never checked...), it is doubtful. FAT32 will NOT work as is.
3. The file MUST exist! (Obvious I know, but had to say it anyways...)
4. The file to be written to MUST be contiguous. Running a defragmenter on the drive or doing a fresh format will acheive this.
5. The file's directory entry must be within the first 16 entries of the root directory. A fresh format of the drive will ensure this.
6. Ensure the file's name is in the old DOS 8.3 format
7. File name is not case sensitive, however the name specificed in the source MUST be uppercase (FAT16 stores all file names in upper case).
8. File's length (in bytes) should be a multiple of 512, data loss in the last sector will occur if this is not followed.
9. File must contain at least one 0x00 character. Use program below to generate a fresh file.

Enough already! Show me the files...

Links to the files are as follows:
- carmon.c - C source code - Updated: April 7, 2004
- carmon.hex - Compiled HEX file - Updated: April 7, 2004
- carmon_schematic.gif - Schematic - Updated: April 7, 2004
- create_wiped_file.c - Creates a file full of zeros of supplied length
- extract_text.c - Updated: April 7, 2004 - Extracts all text up to the first 0x00 character to specified file
- proc_gps.zip - Updated: April 7, 2004 - Perl script that seperates a GPS log file into seperate "trips"
- proc_gps - Simple shell script that invokes the "extract_text" program followed by "proc_gps.pl"
- GPS client project directory - GPS client that outputs to a serial port the important GPS sentences at intervals ranging from realtime to 64 times realtime.
  Connect the serial port to a computer running a GPS program to see the playback mapped to actual positions.
  Compiles with Borland C Builder Ver. 3.0, however should be modifiable to other versions with little effort.

And now, some pretty pictures

Image of the prototype
Schematic, please click on image to open full res version
Screen capture of the debug output during test writing
Screen capture of the the program extract_text being run
Screen capture of a sector view of the last sector written
Screen capture of GPS client
Pictures of the final hardware

NEW! - The GPS "Mouse" mounted in car
NEW! - The GPS "Mouse" underside
NEW! - Carmon mounted in glove compartment

Miscellanous/Loose Ends

The project is now in the car.

The project does work in all conditions it's been exposed to, except for one day when it hit -30C in the car, it didn't start up until it had warmed up a little.
I believe the problem was the extreme low temperature preventing the oscillator from starting up correctly.

I know for a fact that it doesn't work at the moment with some hard drives.
Since I only plan on using two identical cards, both of which work fine, I'm not planning on investigating it further.
My guess is it's just timing related issues that are causing the failures.

Latest Notes/Updates

April 7, 2004 - The project has been promoted to my "completed" section.
It has been operating perfectly for a few months now.
I've purchased a new GPS for permanent mounting in the car, it looks like a computer mouse! :)
I have made a few tweaks to the PIC code and the scripts used, please download the new versions above.
I also have a few new pictures of the system actually installed in the car.

January 18, 2004 - The project has been in the car for the past three weeks.
Aside from a fail to boot on one very cold morning, (the car almost didn't start either!) it has not shown any further problems.
After some more testing I will place this project in the "completed" section!
Also added some New Files and some New Pictures

December 19, 2003 - Added some minor code fixes (problem detecting EOF for files larger then 16MB sector boundary).

November 18, 2003 - Bug appears to be fixed! :)

The bug I had, where the card wouldn't be written to past 16MB seems to be fixed.
As described before, the bug was due to the routine set_sector which does the LBA to CHS translation.
That function used the function div(), which only accepts signed integers as it's parameters.
Since the sector being written to above the 16MB barrier is out of the range of a signed integer, the drive was seeing an invalid CHS combination, resulting in a hang.
I tried to use the function ldiv() which is capable of dealing with long integers, however despite my best efforts I was unable to make it fit within the limitations of PICCLite.
In the end I wrote my own division code (using for loops) and that appears to be working!
I'm again doing some long term testing to ensure things are correct. After that's done I'll finish with the hardware and mount it in the car! :)
As suggested by a person I work with I plan to have some results posted here when I do finally get it going "live"!
Anyways, I've updated both the files carmon.c and carmon.hex in case anyone wants to have a look at the updated version. I've commented out much of the "debug" code that isn't really necessary.

Please check back from time to time for updates.

Resources

Obviously, to do a project like this requires quite a few resources, below are a few I used:

PICLIST - The PICLIST is a email based mailing list consisting of nearly 2000 people. Think of it as a PIC centered think-tank: a group of individuals that have probably seen most of the problems encountered while working with PICs and are eager to help others in solving those problems. I'm a co-admin of the list.
Microchip.com - The manufacturer of the PIC Microcontroller line. A great product, a great company. Can't really do anything with a PIC without referencing one of their datasheets or app notes.
Adi's Electronics - So, you've got a CF card, how do you connect to it? I used a product from this company, it lets you connect a CF card to a standard IDE cable. Very fast shipping, great customer service. Highly recommended. 50 tiny pins = bad. 40 0.1" spaced pins = good. Need I say more?
Wesley's PIC Pages: IDE Controller - Excellent piece of work by Peter Faasse. While he connected a hard drive to an 8051, the information was more then enough to do the same with a PIC.
Microsoft: FAT32 File System Specification - While I'm certainly not much of a fan of Microsoft (in fact my opinion of them isn't the best), they are the best resource for anything dealing with the FAT spec. While this spec details FAT32 it also covers FAT16 and FAT12 (there isn't much difference between them on the really low level). The info in this one document was all I needed, so thank you for that Microsoft.
HI-TECH - Manufacturer of the PICCLite C compiler used for this project. Kudos to them for releasing a version of their compiler for free use. If I were doing this sort of thing more, (and had the money) I'd likely buy their full version. A direct link to PICCLite C can be found here (It's kind of hard to find if you don't know where to look).
WinHex - An amazing program that allows you to look at hard drives on a sector by sector basis. Was invaluable during debugging of this project
Laipac - The company I bought my GPS unit from (the G10 RS232). They've got a bunch of cool stuff and are great to deal with. Highly recommended.

Question? Comments?

If you have any questions about the project or comments feel free to email me at: webmaster@farcite.net

Please note:

All contents on this site remain the property of myself.
Please feel free to use the information found here in "one off", personal, hobbiest type projects.
No permission is granted for using the information on this site in commercial type projects.
Please contact me at webmaster@farcite.net about using any of the contents on this site in a commercial application.

Copyright 2003, Herbert Graf. All rights reserved.

Back

Last update: Sunday, January 18, 2004