Wednesday, September 16, 2009

The newline legacy

In a recent post, I talked about a legacy technology from the 1800s that's an integral part of hundreds of millions of computers today: the QWERTY keyboard layout. QWERTY was designed as an usability antipattern, and its widespread use probably costs the U.S. economy a billion dollars a week in lost productivity. That's my SWAG estimate, anyway.

But that's a hardware problem. ;^)

As a programmer, I think the legacy annoyance I most love to hate is the newline.

The fact that the computing world never settled on an industry-standard definition of what a newline is strikes me as a bit disconcerting, given how ubiquitous newlines are. But it's way too late to change things. There's too much legacy code out there, on OSes that aren't going to change how they treat newlines. The only OS that ever changed its treatment of newlines, as far as I know, is MacOS, which up to System 9 considered a newline to be ASCII 13 (0x0D), also known as a carriage return (CR). It's now the linefeed (ASCII 10, 0x0A), of course, as it is in most UNIX-based systems.

It always bothered me that DOS and Windows adhered to the double-character newline idiom: 0x0D0A (CR+LF). To me it always seemed that one character or token (not a doublet) should be all that's needed to signify end-of-line, and since UNIX and Linux use LF, it makes sense (to me) to just go with that. But no. Gates and company went with CR+LF.

Turns out it's not Gates's fault, of course. The use of CR+LF as a newline stems from the early use of Teletype machines as terminals. With TTY devices, achieving a "new line" on a printout required two different operations: one signal to move the print head back to the start position, and another signal to cause the tractor-feed wheel to step to the next position in its rotation, bringing the paper up a line. Thus CR, then LF.

The fact that we're still emulating that set of signals in modern software is kind of funny. But that's how legacy stuff tends to be. Funny in a sad sort of way.

In any event, here's how the different operating systems expect to see newlines represented:

CR+LF (0x0D0A):
DOS, OS/2, Microsoft Windows, CP/M, MP/M, most early non-Unix, non-IBM OSes

LF (0x0A):
Unix and Unix-like systems (GNU/Linux, AIX, Xenix, Mac OS X, FreeBSD, etc.), BeOS, Amiga, RISC OS, others

CR (0x0D):
Commodore machines, Apple II family, Mac OS up to version 9 and OS-9

NEL (0x15):
EBCDIC systems—mainly IBM mainframe systems, including z/OS (OS/390) and i5/OS (OS/400)

The closest thing there is to a codification of newline standards is the Unicode interpretation of newlines. Of course, it's a very liberal interpretation, to enable reversible transcoding of legacy files across OSes. The Unicode standard defines the following characters that conforming applications should recognize as line terminators:

LF: Line Feed, U+000A
CR: Carriage Return, U+000D
CR+LF: CR followed by LF, U+000D followed by U+000A
NEL: Next Line, U+0085
FF: Form Feed, U+000C
LS: Line Separator, U+2028
PS: Paragraph Separator, U+2029

For more info on newlines and edge cases involving newlines, the best article I could find on the web is this one by Xavier Noria. (It's quite a good writeup.)

There's also an interesting discussion of newlines in the ECMA 262 [PDF] specification. See especially the discussion on page 22 of the difference in how Java and JavaScript treat Unicode escape sequences in comments. (For true geeks only.)

Many happy returns!


  1. Newlines are marginally annoying, but hard-wrapped text is worse IMO because it's harder to fix automatically.

  2. Are you interested in writing qualitative research? Here you can read more information about it

  3. Therefore, the most crucial thing to look out for when searching for Hyderabad packers and movers is their level of reliability. However, it’s not only difficult to find best packers and movers Hyderabad but also to differentiate the right ones from the wrong guys. The ideal solution to avoid such a situation is to search for packer and movers Hyderabad within your budget and do thorough research about each company you interact with, before hiring them.

    Packers and Movers Hyderabad
    Packers and Movers in Kondapur
    Packers and Movers in Gachibowli
    Packers and Movers in Kukatpally

  4. There are many packers and movers in Hyderabad who are looking for ways to make easy money from innocent families and individuals like you who are planning to relocate. Such packers and movers Hyd may either provide extremely bad service and end up damaging your goods permanently, or disappear with your entire household items instead of delivering them to the destination.
    Movers and Packers in Hyderabad
    Movers and Packers in Jubilee Hills
    Movers and Packers in Banjara Hills
    Movers and Packers in Kondapur

  5. Thanks for your stop at Indian Packers and Movers in Mumbai. We really know the intentions of the people who are moving their homes or offices in Mumbai.
    Packers and Movers in Kharghar
    Packers and Movers in Dombivli
    Packers and Movers in Thane West

  6. We provide a wide range of computer chairs, ergonomic chairs, leather chairs and thus, we are leading manufacturer of office furniture chairs. We supply products not only to local clients but also to national and international clients.
    Chair Manufacturers in Mumbai
    Chair Supplier in Mumbai
    Office Chair Supplier in Mumbai
    Visitor Chair Supplier in Mumbai
    Chair Dealers in Mumbai

  7. Indian Packers and Movers in Mumbai give packing and moving services like loading & unloading, packing & unpacking, car carriers, transportation, domestic &local shifting, international shifting, Industrial shifting.
    Packers and Movers in Thane
    Packers and Movers in Kalyan

    Packers and Movers in Mumbai
    Packers and Movers in Powai
    Packers and Movers in Vashi
    Packers and Movers in Chembur
    Packers and Movers in Kharghar

  8. you have a good knowledge of Cricket you Can play daily online ,live cricket games,
    cricket online games, best cricket games, play cricket online, cricket game online at best cricket games app for android phones to win real
    cash and amount to instant approve in your bank or Paytm wallet.
    play cricket online


Add a comment. Registration required because trolls.