Reenigne blog

This was a question which puzzled me when I first found out about it, but now that I understand all the history behind it, it makes perfect sense.

The IBM PC (5150) was originally designed output to an NTSC television in mind - hence the 4.77MHz clock speed (4/3 the NTSC color carrier frequency - allowing the video output and CPU clock to share a crystal). It was thought that home users would generally hook their PCs up the TV rather than having a separate, expensive, monitor. Another major limiting factor in the design of the CGA was the price of video memory - the 16Kb on the card would have been fairly expensive at the time (it was as much as the amount of main memory in the entry level PC). TV resolution is 912x262 at CGA 2-colour pixel sizes in non-interlaced mode, but TVs (especially CRTs) don't show all of the image - some of those scanlines and pixels are devoted to sync signals and others are cropped out because they would be distorted due to the difficulties of approximating high frequency sawtooth waves with high-voltage analog circuitry. So 320x200 4-colour and 640x200 2-colour packed pixel modes were chosen because they were a good fit for both 16Kb of memory and TV resolutions.

That system did work quite well for many home users - lots of CGA games have 16-colour composite output modes. But it wasn't so good for business users. These users tended not to care so much about colour but did care about having lots of columns of text - 80 was a common standard for interfacing with mainframes and for printed documents. But 80-column text on a TV or composite monitor is almost completely illegible, especially for colour images - alternating columns of black and white pixels in a mode with 320 pixels horizontally gets turned into a solid colour by NTSC. So for business users, IBM developed a completely separate video standard - MDA. This was a much simpler monochrome text device with 4Kb of memory - enough for 80 columns by 25 rows of text. To display high quality text, IBM decided on a completely different video standard - 370 scanlines (350 active) by 882 pixels (720 active) at 50Hz, yielding a 9x14 pixel grid for high-fidelity (for the time) character rendering. In terms of timings, the character clock is similar (but not identical) to that of the CGA 80-column text mode (presumably 16.257MHz crystals were the closest they could source to a design target of 16.108MHz). To further emphasize the business target of the MDA card, the printer port was built into the same card (a printer would have been de-rigour for a business user but a rare luxury for a home user). Business users would also usually have purchased an IBM 5151 (green-screen monitor designed for use with MDA) and IBM 5152 (printer).

CGA also had a digital TTL output for displaying high quality 16-colour 80-column text (at a lower resolution than MDA) on specially designed monitors such as the IBM 5153 - this seems to have been much more popular than the composite output option over the lifetime of these machines. The two cards used different memory and IO addresses, so could coexist in the same machine - real power users would have had two monitors, one for CGA and one for MDA (and maybe even a composite monitor as well for games which preferred that mode). The 9-pin digital connectors for CGA and MDA were physically identical and used the same pins for ground (1 and 2), secondary intensity (7), horizontal sync (8) and vertical sync (9) but CGA used 3, 4 and 5 for primary red, primary green and primary blue respectively whereas MDA used pin 7 for its primary video signal. MDA also used a negative-going pulse to indicate vertical sync while the CGA's vertical sync pulse is positive-going.

So for a while these two incompatible standards coexisted. The next major graphics standard IBM designed was the EGA, and one of the major design goals for this card was to be an upgrade path for both home and business users that did not require them to buy a new monitor - i.e. it should be compatible with both CGA and MDA monitors. This was accomplished by putting a 16.257MHz crystal on the card and having a register bit to select whether that or the 14.318MHz one would be used for the pixel clock (and by having the on-board video BIOS ROM program the CRTC appropriately). By 1984, it was not out of the question to put 128Kb of RAM on a video card, though a cheaper 64Kb option was also available. 64Kb was enough to allow the highest CGA resolution (640x200) with each pixel being able to display any of the CGA's 16 colours - these would have been the best possible images that CGA monitors such as the IBM 5153 could display. It was also enough for 4 colours at the higher 640x350 resolution - allowing graphics on MDA monitors. With 128Kb you got the best of both worlds - 16 colours (from a palette of 64) at 640x350.

IBM made a special monitor (the 5154) for use with the EGA. This monitor could display both 200-line and 350-line images (deciding which to use by examining the vertical sync pulse polarity), and allowed users would be able to take advantage of all 64 colours available in 350-line modes. The video connector was again physically the same and pins 1, 3, 4, 5, 8 and 9 had identical functions, but pins 2, 6 and 7 were repurposed as secondary red, green and blue signals respectively, allowing all 64 possible colours. But they wanted this monitor to be compatible with CGA cards as well, which meant that in 200 line mode it needed to interpret pins 3-6 as RGBI instead of RGBg and ignore pins 2 and 7. So even with a 5154, the EGA needed to generate a 4-bit signal when connected to a CGA monitor, disabling pins 2 and 7.

I guess the designers thought that sacrificing 48 of EGA's colours in 200-line modes was a small price to pay for making the EGA monitor compatible with CGA cards. Presumably they thought that if you had an EGA card and an EGA monitor you would be using 350-line mode anyway, or be running legacy CGA software which wouldn't miss those extra colours.

One thing I haven't mentioned here is the PCjr graphics. For the purposes of the discussion above it's essentially the same as CGA (it has the same outputs) but it's more flexible and slower due to the use of system RAM as video RAM, as many 8-bit microcomputers did in the 80s.

For a while now I've been wanting to write a cycle-exact emulator for the original IBM PC and XT machines that are the direct ancestors of the machine I'm writing this on (and, almost certainly, the machine you're reading this on). I've written an 8088 emulator with cycle timings that I think are plausible but I have no idea if they're correct or not. The exact details of the timings don't seem to be published anywhere (at least not on the internet) so the only way to determine if they are correct is to compare against real hardware.

So, when I saw a cheap XT for sale on eBay recently, I made a spur-of-the-moment bid, and won it. It is a bit beaten up - the case was bent in one place, the speaker cone was falling off and the end of one the ISA slots was broken off. All these problems were easy to fix with a hammer and a bit of superglue, though. It lacks keyboard and monitor, which makes it rather difficult to do anything useful with it, but it did come with all sort of weird and wonderful cards:

• Hyundai E40080004 MDA card with 66Kb of RAM. It's all discrete logic apart from the CRTC, RAM and a ROM which I think is a character ROM rather than a BIOS ROM (though I could be wrong). The amount of memory makes me suspect it can do graphics, but I can't find any documentation - I'd probably have to reverse engineer a schematic to find out exactly what it's capable of.
• Tecmar Captain 200044 card with 384Kb RAM (bringing total to 640Kb), serial port, connector for a parallel port and possibly an RTC (it has a CR2032 battery on it anyway).
• AST 5251/11 - apparently this enables PCs to connect, via Twinax, to a 5251 terminal server. It has the largest DIP packaged IC I've ever seen - a Signetics N8X305N RISC Microprocessor running at 8MHz (16MHz crystal) with 6KB of ROM and 96Kb16KB of SRAM (12KB on a daughterboard) for its own purposes.
• A generic serial+parallel card.
• IBM floppy and MFM hard drive controller cards.
• IBM serial card.
• PGS scan doubler II. It seems to be pretty rare, there's only one mention of it on Google and I've never heard of such a device being used with PCs before (though I understand they were more popular in the Amiga-verse). It's only uses the power and clock lines from the ISA bus - it has two CGA-style ports on the back, one for input and one for output. You loop the CGA card's output back into one of the ports, and the other one outputs a signal with twice the line rate (it buffers each line in its internal 2Kb RAM and outputs each one twice, at double the pixel rate). I'm guessing the output had to be a monitor with a CGA input which could run at VGA frequencies, which can't have been all that common.

It also comes with a floppy drive and a hard drive which makes the most horrendous metal-on-metal grinding sounds when it spins up and down (so I'll be very surprised if it still works).

This machine has clearly been a real working PC for someone rather than a perfectly preserved museum piece - it tells stories. At some point the original MDA card failed and was replaced with a clone, the RAM was upgraded, it's been used as a terminal for a mainframe, and at some point the owner read about this device that improves your video output, bought one, found it didn't work with his MDA card but just left it in there.

Due to lack of keyboard and monitor (neither my TV nor my VGA monitor can sync to MDA frequencies) I haven't got it to boot yet. I tried to use the scan doubler with the MDA and my VGA monitor (connecting the mono video signal via the red channel of the doubler) and the monitor was able to sync but the output was garbage - I guess the doubler is either broken or only works with CGA frequencies. If it does work with CGA then it'll be useful for seeing the TTL CGA output (though I'll have to put something together to convert the RGBI digital signals to analogue RGB - with proper fix up for colour 6 of course).

I ordered a CGA card but decided to see if I could jerry-rig something up in the meantime. I programmed my Arduino to pretend to be an XT keyboard and also the "manufacturing test device" that IBM used in their factories to load code onto the machine during early stage POST (it works by returning 65H instead of AAH in response to a keyboard reset). I then used this to reprogram the CRTC of the MDA to CGA frequencies (113 characters of 9 pixels at 16MHz pixel clock for 18 rows (14 displayed) of 14-scanline characters plus an extra 10 scanlines for a total of 262 scanlines). The sources for this are on github.

Next, I connected the hsync, vsync, video and intensity lines to a composite output stage like the one in the CGA card (it's just a transistor and a few resistors) and put some junk in video memory. Amazingly, it works - there is a barely legible picture. Even more amazingly, it is in colour! On the same breadboard I was doing this on, I had a Colpitts oscillator driving a 3.5795MHz (NTSC colour burst frequency) crystal from an earlier experiment (before my XT arrived I was trying to get colour TV output from my Arduino). This oscillator wasn't connected to anything, but the very fact that that frequency was bouncing around nearby was enough to turn off the colour killer circuit in the TV and allow it to interpret certain horizontal pixel patterns as colours.

The colours themselves aren't actually related to the picture in any useful way - the hsync and crystal aren't in sync so the phase (and hence the hues) drift over time. In fact, by applying slight heat and/or pressure to the crystal with my fingers, I can change the frequency slightly and make the hue phase drift rate faster or slower with respect to the frame rate, and even stop it altogether (though it's still not a multiple of the line rate so the colours form diagonal patterns).

The sync isn't quite right - because the hsync and vsync signals are just added together the TV loses horizontal sync for 16 lines or so during vsync and then spends half the picture trying to recover. Unfortunately the CRTC in the MDA card has a vertical sync pulse height fixed at 16 lines but it needs to be closer to 3 for NTSC so I haven't been able to get a stable signal even by XORing the signals like the CGA does. The CGA uses a 74LS175 to get a 3-line sync signal, but I don't have one of these to hand.

Here's the schematic for the circuit as I would have made it if I had the parts:

Unfortunately I haven't been able to continue the BIOS POST sequence after running this code - I tried jumping back into the BIOS at a couple of places but it just froze. I'll have to tinker with it some more to see if I can get it to work and determine where the POST is failing next.

I've determined that it should be possible for the XT to send data back over the keyboard line (the clock line is controllable by software). So I'm planning to do bidirectional communication between the XT and a host PC entirely over the keyboard port! I'm writing a tiny little OS kernel that will download a program over the keyboard port, run it, send the results back and then wait for another one.

Unfortunately my plans have been derailed because the power supply unit has failed. I noticed that the PSU fan wasn't spinning - I think that has caused some parts in the PSU to overheat. One resistor in there looked very burnt and the resistors for discharging the high-voltage smoothing capacitors were completely open circuit. I replaced all these but it's still not working. I've ordered a cheap ATX power supply and will be transplanting the guts into the XT PSU's box so that I can keep the big red switch.

My software TV/monitor emulator is best thought of as a bunch of filters for transforming a signal in certain ways, connected together in a pipeline:

• Decoding composite signals to YIQ
• Transforming YIQ signals to RGB
• Horizontal rescaling
• Ghosting due to signal reflection in the cable
• Adding noise

Because that's the best way to think about it, that's how I'd like to implement it. Then it will be easier to remove/replace filters that I don't need or that I want to implement in a different way. The filters.h file in the crtsim source implements this architecture.

When you have a pipeline, there are two ways to drive it. One is "consumer pulls" and the other is "producer pushes". In this case, the consumer is the code that actually renders the window to the screen. In "consumer pulls" mode, this code will fire at probably 60 times per second (potentially depending on the refresh rate of the monitor on the host machine) and each time it does, it will ask the filter which supplies its data for enough data to render one frame (or field, if we're doing interlaced signals). This filter will then in turn ask the next one along the chain for data and so on up the chain until we get to the code that actually generates the composite signal.

In "producer pushes" mode, the producer generates data at a constant rate (possibly fixed by some other source of time in the system such as the audio device outputting at the correct rate). This data is then pushed from each filter to the next in the chain until it gets to the consumer. When the consumer has collected enough data to render a frame, a frame is rendered.

So for the purposes of emulating a TV or monitor as part of a microcomputer system emulator, "consumer pulls" and "producer pushes" modes can be thought of as "video rate driven" and "audio rate driven" modes respectively. Most emulators are hard-coded to do one or the other. But which one is best is determined by the user's hardware and what they're doing with the emulated system (video driven mode will generally smoother graphics while audio driven mode will generally give more stable audio). So ideally we'd like to make the choice user-selectable.

A third possibility is for the producer code to decide when to draw a frame and to call for a window redraw, which causes a data pull through the filter chain. However, I've discounted this idea because that is an incorrect placement of responsibility. The producer doesn't (and shouldn't) know about the state of the monitor. Even if it has just produced a vsync pulse it doesn't necessarily mean its time for a new frame (if the monitor is "rolling" as it will do momentarily when the signal timebase changes, it won't be).

There is another factor in pipeline design which is how the data stream corresponds to function calls. The simplest way would be to have each sink call the corresponding source each time it needs a sample (in pull mode) or each source call its corresponding sink each time a sample is available (in push mode). However, there are potentially quite a few filters and (because they are all replaceable at run time) each call from one filter to the next will be a virtual function call. That means that the compiler can't inline the code and the CPU's pipelining will get screwed up. According to this one can expect a virtual function call overhead of maybe 13 nanoseconds compared to inlined code (a crude test, but sufficient for order-of-magnitude calculations). Since most of our samples will be at 14MHz (4 times the NTSC color carrier frequency) that's only about 5 virtual function calls per sample before we've used up all our CPU.

So each function call really needs to transfer a pile of samples, not just one, and we will need to have circular buffers in between the filters to keep the data. How many samples should we transfer at once? A good rule of thumb for figuring that out is that one's hottest code and data should fit in L1 cache (which is maybe 64Kb on modern CPUs). Divide that up by the number of steps in the pipeline and we're looking at low-single-digit numbers of Kb to be passed at once. A scanline's worth of data (910 samples, give or take) is probably about right. That reduces the virtual function call overhead by three orders of magnitude which puts it well into the negligible range. Conceivably one could try benchmarking with lots of different "samples per call" values and then pick the one with the best overall performance (taking into account both call overhead and cache misses). I'll probably do this at some point.

One disadvantage of the pipeline architecture is that it introduces some variable amount of latency - not enough to normally be visible to end users, but this does complicate one thing that I want to emulate - light pens. A light pen is just a fast light sensor that can be placed anywhere on the screen. When the electron beam passes underneath it, it sends a signal to the computer. The computer knows where the beam is supposed to be at any given moment, so it can figure out where the light pen is. However, for an emulator to have proper lightpen support, it needs to have very low latency between the screen and the machine emulation. For this reason, I might abandon the pipeline architecture and just hard-code all the signal munging effects I care about in the CRT simulator itself, processing a line at a time and stopping when the horizontal sync pulse is found. Then, if the lightpen is anywhere on the next line the CRT will be able to tell the machine emulation exactly when the lightpen is going to be triggered.