Betacantrips/ Bits

Little things go here. There's also an archive.

free software audio

Producing music with Free Software isn't all that hard, but as with other Free Software you need to know what you're doing. I don't. But I've learned a bit in wandering around and seeing what I can do. This article is meant to give an overview of what I think I know.

Samples or notes

Audio software can take one of two views of the audio it works with. One, the audio can be a very long and complex waveform, represented by a long list of samples. An audio CD is like this -- each second, 44100 samples are read off of the CD and played. Sample-based file formats are WAV, MP3, OGG. The variation in these samples creates pressure on air, which creates vibrations, which creates music. Two, the audio can be notes, which are "played" in some way by the software. If you've ever seen an antique player piano, it has a note-based view of music. Some device reads off notes and, depending on each note, takes some kind of action (presses a different key, or something). Note-based file formats are MIDI, IT, MOD.

Speeding up waveform-based audio will increase the frequency of the vibrations, and thus the pitch as well as the tempo. Imagine spinning a record around with your finger, faster and faster. Speeding up note-based audio only increases the tempo. I used to have a jack-in-the-box which played "Pop Goes the Weasel" when you cranked it -- cranking it faster made it play faster. If you ever had something like that, imagine that.

If you're working with waveform-based audio, your first choice is likely to be Audacity. If you're working with note-based audio, your first choice is likely to be Rosegarden.

Waveform-based audio

Waveform-based programs are good for recording from a mic, or doing postproduction on different tracks, but they're not optimal for creating sounds from scratch or composing music, so I'll skip over them.

Suggested tags: sound::waveform

Note-based audio

In note-based audio, notes are entered in some way, and then some software goes through the list of notes and plays them. Sometimes playing is done in the same program that edits the notes; sometimes it's done in a separate program.

Terms:

  • sequencer: "a device or piece of software that allows the user to record, play back and edit musical patterns" (Wikipedia).

Suggested tags: sound::notes. There's already a works-with::music-notation, but to me this suggests a staff and clefs and things like that, which isn't true for programs like nyquist, which don't use any musical notation at all. Alternately, sound::sequenced or sound::MIDI might be more concise, but to me sound::notes encompasses all of the concepts that sound::sequenced and works-with::music-notation leave out.

Editing notes

Notes are arranged in patterns, which can be moved around, copied, or re-used, depending on the program. Editing a pattern is generally done in one of the following interfaces.

Notation

Music notation, with staffs, clefs, bars, and all the assorted musical notes. Classically-trained musicians might recognize and appreciate this kind of notation, but new musicians might not like it.

../support/rosegarden-notation.png

Suggested tags: sound::interface:musical-score or maybe sound::interface:notation?

Piano roll

Piano roll interfaces are like a big grid. On the left side of the grid, there are all the keys of a piano. Each key on the piano is aligned with a row of the grid. The grid extends to the right. Each note is shown as a rectangle on the grid, with its position showing both its pitch and its time, and its width showing its duration.

../support/rosegarden-pianoroll.png

Suggested tags: sound::interface:pianoroll

Tracker

Originally, the term "tracker" applied to a certain kind of electronic music program, meant for editing certain tracked music formats: MOD, XM, IT. These formats are meant for sample-based synthesis (see below) and have a certain distinctive row-based interface. The interface is general enough to apply to other programs -- shaketracker is a MIDI sequencer that uses this interface style.

A tracker interface looks like a screen full of numbers. Each row of numbers represents a note. Time goes down. The numbers in each column has a different meaning.

../support/aldrin-tracker.png

(A screenshot of Aldrin's tracker mode. Conceptually the unused columns are for envelopes, pitch bends, and so forth, but I couldn't make any of them work.)

Suggested tags: sound::interface:tracker

Tracker software

FastTracker 2 is a well-known piece of electronic music software from the early 90s. It's got a tracker interface and does its own sample-based synthesis. Soundtracker is a FastTracker clone.

ImpulseTracker is another DOS-era tracker. The .IT format comes from this tracker. SchismTracker and Cheesetracker are both clones of IT.

Suggested tags: sound::tracker-clone (if this is even necessary)

Combining

Once you have some patterns, you lay them out. Each program has different ways of doing this.

Synthesis

Once you have notes, you have to convert them to sounds. This is the domain of synthesizers. Some programs do both note-based editing and synthesis, like Aldrin, Beast, or LMMS, but there are also separate synthesizers like Fluidsynth, TiMidity, and ZynAddSubFX which can do synthesis for other programs.

Suggested tags: sound::synthesis

Sample-based

In sample-based synthesis, a sound file in WAV or other format is made for the instrument, and this "sample" is pitch-shifted to make different notes. Sample-based synthesizers often have options like using only part of the sample, looping it, etc.

Suggested tags: sound::synthesis:sample-based

Others

There are other programs that do other forms of synthesis.

  • ZynAddSubFX: generic "software synthesizer", capable of producing many different kinds of sounds.
  • OM: "modular synth" in which components are connected to create a "synthesizer".
  • Aeolus/horgand: synthesizers that emulate the sound of a pipe organ.

Communication

Some protocols and acronyms:

  • MIDI channels. jackd handles routing MIDI messages. This is most common when talking to hardware synthesizers, or getting input from hardware MIDI devices. MIDI events include "the pitch and intensity of musical notes to play, control signals for parameters such as volumue, vibrato and panning, cues and clock signals to set the tempo." (Wikipedia.)
  • LADSPA. An API that is used by audio plugins; lots of programs support it (audacity does, and so does OM, in very different ways), and there are a ton of plugins (in blop, caps, cmt, fil-plugins, mcp-plugins, swh-plugins, and tap-plugins). LADSPA plugins generally transform audio in some way.
  • DSSI. "DSSI is an API for audio plugins.. It may be thought of as LADSPA-for-instruments, or something comparable to VSTi." (dssi homepage) Much like MIDI, information regarding what notes to play is sent to a DSSI plugin; however, unlike MIDI, the application sending the notes, gets the waveforms back, so that effects can be done on the resulting sounds. [This should be possible to do using OM, but I haven't actually successfully done it yet.] Note, however, that using jack, you can connect the output of one program to the input of another program, and do similar things like that. This may be why DSSI hasn't taken off quite so much.
  • ll-scope: a DSSI plugin which displays audio via an oscilloscope view
  • jack-rack: LADSPA effects plugins, hooks up with jack; "turns your computer into an effects box"

Tags: Are audio::ladspa and audio::dssi useful here? Not sure.

Realtime kernel support

Most audio programs communicate using jackd, which is a sound server meant for low-latency performance and arbitrary connections. Using some client (for example qjackctl), you can connect arbitrary ports on programs to other ports on other programs. In this way, you can route the output of your synthesizer through other effects plugins. jackd is cool but it's not used much outside of professional audio, so you likely haven't used it before. The number one stumbling block to getting jackd to work for you is setting up realtime support in your kernel.

Realtime support is actually not "hard" realtime support, as you might find in QNX. In this context, realtime just means "low latency". Low latency is important, because jackd wants to send samples around your computer at 44100 times per second (or whatever), and if it can't get a sample in on time, it'll sound bad.

Low latency support has two aspects: PREEMPT_RT, and realtime-lsm. PREEMPT_RT is a patch, maintained by Ingo Molnar and various others, which makes the kernel "fully preemptible", i.e. the kernel can be interrupted in many more circumstances than it used to. This supports lower latency by reducing the amount of time jackd will have to wait to interrupt the kernel. realtime-lsm is a module that grants extra permissions to processes, like jackd, that want priority on computing time. See LWN for more information on realtime support in Linux.

PREEMPT_RT is slowly being folded into the mainline kernel, and it may be that by the time you read this, kernels will by default support preemption. But if it doesn't, you'll have to:

  • grab the PREEMPT_RT patch from http://rt.wiki.kernel.org/
  • apply it
  • make sure your config is correct
  • build kernels as you ordinarily do

realtime-lsm is available as a module, and on a Debian system you can apt-get install realtime-lsm, and use module-assistant to build it.

Posted Mon Sep 3 10:21:28 2007

"Reply-to munging considered harmful" considered infuriating

I'll admit right out that I'm "new" to email (meaning that I've only been using it for ten or twelve years, never implemented an MUA or an MTA, and don't administrate any mail servers), but there's this meme titled "Reply-To" Munging Considered Harmful (started by one Chip Rosenthal) that I would like to take issue with. The meme has managed to worm its way into mailing list administrators worldwide, and I even thought I would be one of them, waving the standard of "well-behaved Internet software" as I marched on. But it has not come to pass.

There are two mailing lists I'm on. One, the pygame-users mailing list, munges the Reply-To header. Another, the pyode-user mailing list, does not. On every mail I have replied to on the pyode mailing list, I have sent a follow-up to the list saying "oops, sorry Person-I-Replied-To for the duplicate mail, this was supposed to go to the list too". This hasn't happened once on the pygame mailing list. Mr. Rosenthal asserts that reply-to munging "adds nothing". This is not true. In my case, it adds relief from embarassing "shit I fucked up with the email thing" notes and copies sent to lists.

In most computer-mediated fora, if someone says something, and you'd like to respond, it is expected that the response will go to the same place that the original did (meaning: sent to the same people). Mr. Rosenthal asserts that any reasonable mailer has two functions, one for "reply" and one for "group reply", and that any idiot user can be taught to press "g" if they want to respond publically, and "r" if they want to respond privately. But a user expects that a "reply" will be sent to everyone relevant; it is the private responses that are special, not the public ones.

Back in the '90s, you frequently found that some AOL user had sent you and 400 other people a message, and one of those 400 other people had responded with "I don't want to read this crap", and another of those 400 other people had written back with "lol", etc., with each message growing longer and more meaningless until you gave up and got a different email account. This was entirely the result of making "reply-all" the default function. Idiot users who didn't know better would send replies to people who didn't need them. So for the most part I think making "reply privately" the default function for end-user software is a defensible choice; it prevents unwanted noise. But when you start a mailing list, you want to make the default reply function (on a per-message basis) "send to list"; it is for this purpose that reply-to munging is helpful.

This is the case for reply-to munging. Let's go through Mr. Rosenthal's arguments against.

  • It breaks the reply-privately function. It's not gone, only more difficult. This isn't always a make-or-break thing; if nobody ever replies privately, then it's a tradeoff that should be considered.

  • Administrators shouldn't dictate that all replies go to the list. This is a straw man. Administrators know best what the tone and flavor of their mailing list is. If it's primarily technical discussion, or a digital gathering space that doesn't usually wander off into private email discussions, then absolutely they should have the power to make replies go to the list by default.

  • Reply-to munging can make it difficult or impossible to find the sender of a message. As someone who is new to email, I'm not familiar with the uses Mr. Rosenthal offers: if someone is sending a message that they haven't authored, typically they indicate this fact in the body of the email and include the content of the message as a quote. If someone is sending email from one address, but would like to recieve email at a different address, they should change the From: header to match the Reply-To: header. (I know it's really more complicated than that, but I think this approach is viable.)

  • Munging reply-to headers penalizes those who use good software. Mr. Rosenthal writes, "If a few people need to type in a full reply address so that everybody else can use all the features of their mailer, I say, 'Fine!'" To this I reply: if a few people cannot use all the features of their mailer so that everybody's replies go sensible places by default, I say "Fine!"

  • Munging reply-to headers makes more work if you want to reply privately. On the pyode mailing list, hitting "Reply" will reply To: the person who sent the mail; hitting "Reply all" will go To: the sender and Cc: the list. Most users (including technically inclined ones) will assume this means that the sender will get a duplicate email from the list, and so will scratch out the To: line and change the Cc: to a To:. So not munging reply-to headers leads to users doing just as much work, assuming they even remember to reply publically. (Otherwise they have to go to their copy of the sent message and send a copy to the mailing list.)

    In fact, Mailman is intelligent enough to not send a duplicate email in this case, but frankly this is a huge surprise to me considering that Mailman sends passwords in plaintext emails. However, if B replies to A via the list L, the reply will be To: A and Cc: L. If C replies to this email, it will be To: B and Cc: A and Cc: L. This is just ridiculous.

  • A user expects a garden-variety reply to be private. This is just not true.

  • A user may expect a reply to be sent privately, only to discover it going out as public, which can lead to embarassment or worse. I have never seen this happen, and it has never happened to me.

  • If we munge reply-to headers, users will demand a third "reply" option which ignores reply-to headers but addresses an email to the sender of a message. This is complicated, but: if you already accept that there is a category of "private" replies and a category of "public" replies, why shouldn't the "private" reply in this case write to the sender? Anyhow, there's already a third type of "reply" called Reply-To-List, and it doesn't work, so I don't see what good Mr. Rosenthal's suggestion is.

In an ideal world, people would never make mistakes, and they would know that their mailer has reply-private and reply-public modes, and would invoke the correct ones all the time. That isn't the world we live in. In a slightly more plausible world, mailers would have a default "reply" mode which intelligently decides who gets a copy, based on who got the original message, whether it's part of a mailing list, and how dumb the user is, and explicit "private" and "public" modes in case the software's guess is wrong. We don't live in that world either. We also don't live in a world where users expect replies to be private by default. Munging reply-to can be a terrible thing, or it can be benign or beneficial. List administrators should not blindly follow Mr. Rosenthal's suggestion, but think about their users and what is likely to be best. For the lists I administrate, I'm enabling munging.

Lastly, email has grown to be a hugely complicated organism, made up of many users, many pieces of mail software, and many standards. We can't throw the whole thing out and start fresh; even if we did, how would replies work, given that many users are prone to noise generation? Reply-to munging is just a patch, but I think it's the best solution we have, and I'm going to use it when it's called for.

Posted Mon Aug 6 13:55:43 2007
combined log format CGI

Fist Farm needed a CGI that generated Combined Log Format for fistfarm.com, which he was working on at the time. It's surprisingly hard to find one that does just that!

He wanted to call it from inside an SSI, but the ideas should be the same.

Here's what I came up with.

#! /usr/bin/perl

$logfile = "log.blah";

use POSIX;
my $time = POSIX::strftime("[%d/%b/%Y:%H:%M:%S +0000]", gmtime);
my $line = sprintf("%s %s %s %s \"%s\" %s %s \"%s\" \"%s\"\n",
                   $ENV {REMOTE_ADDR} || "-", # %h
                   $ENV {REMOTE_IDENT} || "-", # %l
                   $ENV {REMOTE_USER} || "-", # %u
                   $time,              # %t
                   "$ENV{REQUEST_METHOD} $ENV{REQUEST_URI}", # %r
                   "-", # %>s  (status code)
                   "-", # %b (size)
                   $ENV {HTTP_REFERER} || "-",
                   $ENV {HTTP_USER_AGENT}) || "-");
open LOG, ">>$logfile";
print LOG $line;
close LOG;

print "Content-type: text/html\n\n";
print "";

Note that it's impossible to guess the status code or the size of the request from a CGI; I just left them blank.

To call using an SSI, I did:

<!--#include virtual="blah.cgi" -->

But Fist Farm recommends:

<!--#exec cgi="blah.cgi"-->

Posted Mon Jun 11 15:27:22 2007
Spring Semester, 2118

Although the Halloween party started at 8, I was in the library working on a paper until 10. I didn't have a costume so I didn't really feel like going, and besides that I didn't know any of the people at the frat that was hosting it. I'd hoped to write another few hundred words that night while everyone else was out partying, but when I found myself alone my thoughts turned, as they usually did, towards me, and how it didn't matter how many words I wrote because I wasn't very smart to begin with. Eventually I decided no matter how long I tormented myself in a study cell, I wasn't going to put any more words down that night, so I went to the party anyhow. As expected, I didn't know anyone there, so I found myself sitting on an old plaid couch nursing a beer and watching people dancing, talking, pairing off and going elsewhere.

When she walked in, of course, she caught pretty much every eye in the room. You could hear the noises that people make when they stop making out because something extraordinary is going on. She was dressed in a full-body powder blue jumpsuit, and her face and hair had been colored to match. Only her lips were red -- the rest of her was that sharp powder blue, marred only by the telepath badge above her left breast. She must have felt all those heads turn towards her, and maybe she even picked up on the lewd sorts of things people think when they've been drinking, but at no point did a blush ever darken those faint blue cheeks.

She gathered up the grace that the pressure of stares can provide. She walked regally over to the fridge, got herself a drink, and sat down in the easy chair next to the couch. She was closer to me now and I could see her jumpsuit better. It had a zipper down the front and loose fabric at the wrists. Her hands were blue too; had she gotten a full-body cosmetic or was she just careful?

I continued to look her over as she glanced about the room. I had no idea whether she was reading minds or merely checking out the scene (this was before the different marks on telepath badges were introduced). She moved slowly, as though careful or thorough with her gaze, and never changed expression. Eventually her gaze came to rest on me.

"Hi," she said. She smiled. "My name is Kimberly, but everyone calls me Kim."

"Hello, Kim," I said. "My name is Nate."

"Hello, Nate." She continued to stare at me, though after a few seconds her smile faded. Eventually I realized that she must be reading my mind, though I didn't feel anything. I waited, silent, for her to finish.

"You're pretty messed up, aren't you, Nate?" she said.

"Yeah," I said, thinking about how glad I was to be at a party, not alone, distracted from my numerous failings.

"I like being in your mind," she purred. She moved to sit next to me on the couch.

"It's a good mind, I like it, it suits me well," I gibbered, dumbstruck by her attention.

A short while later, we paired off and went elsewhere ourselves. It turned out to be a whole-body cosmetic after all.

--

I didn't realize she was in my section of Social Dynamics until the next day. As I entered the discussion room, she smiled and waved at me. She still had blue hair, but she had gotten rid of the rest of the cosmetic. I'm pretty sure I blushed and waved back. I sat in the open seat behind her.

"Hi Nate!" she giggled, leaning her head back to look at me upside-down. "Great party last night, huh?"

"Hi Kim," I said. "I must admit I wasn't paying too much attention to the party." She giggled at that too. She was very pretty when she giggled. "The TA just sent out our grades on the last test -- how did you do?" When I asked that, her face lost all traces of mirth. I wasn't an esper myself, but the serious look didn't bode well for her grades. She righted herself and turned around in the seat.

"I'm kind of embarassed about it, actually," she said.

"Say no more," I assured her, ever the charmer. "You know, maybe we could have a study session some time."

"I don't know," she said. "I don't really like studying, you know?" She giggled again. I wondered if there was anything behind that pretty giggle. "But maybe we can go for coffee instead."

"Well, Kim, that sounds like fun. Maybe when finals come around I can talk you into that study session."

"I doubt it!" she said, and tossed her blue hair. "I'm very stubborn, you know."

Just then the TA came in, a smile on his face and a spring in his step. He casually went over some of the mistakes he'd found in the papers we'd turned in. He exuded a sense of superiority, but it had been worth it to see him one-foot-in-the-grave last week, when he was still grading. He said someone in the class had gotten an A-plus on the paper, but I found that hard to believe. As he rambled sonorously, I looked at the back of Kim's head. She was staring off to one side, presumably into the middle distance. If she hadn't been doing well in class, why wasn't she paying better attention? Maybe I had hooked up with a ditz. After all, she was certainly pretty enough. Not that you had to be pretty to be a ditz, I thought quickly. This was college, and we were all just trying to look sophisticated. I was urbane enough to allow for the possibility that a ditz could be ugly.

What have you gotten yourself into, Nate? I thought.

--

The next day I met for lunch with my friend Connie.

"You should break up with her," she said, as soon as I mentioned Kim.

"We just went for coffee," I protested. "What, are you afraid she's gonna take over my body or erase my memories or something like that?"

"Don't be ridiculous," Connie said. "But you should break up with her."

"I'm not even dating her yet!" I tried to describe what had happened at the party and after class as best as I could. "We just stayed and talked. Although, to be honest, I get the feeling she's not very bright. Whenever I mentioned classes, she clumsily changed the subject."

"You need to break up with her," Connie repeated. "She can read your mind! Doesn't that scare you? She knows all your dirty little secrets, and she's never even been to your apartment."

"I don't know, Connie. Maybe it means she'll be able to understand me, without all those stupid words that get in the way. Don't you want to be understood by whoever you're dating?"

"You know," Connie said, looking thoughtful, "I had a friend back home who was a lesbian. I thought, wow, that's gotta be so great. Your partner isn't some insensitive guy with a different sex drive than you. It's some woman who knows exactly what you're going through, biology and all. But I asked my friend, hey, do you ever fight? Or do you just understand each other all the time? And she gave me this look, right? and said, oh, we fight from time to time. And when we fight, it's no holds barred. We fight for blood. And we can hurt each other so much worse, because we understand each other." She gave me a withering look, the look her friend must have given her. "Listen, I think it's great you're dating, you know, a telepath. And I don't go around throwing rocks and calling anyone names or anything like that. But I just think a relationship has its secrets, and maybe there are some things you don't want her to know. I'm not just talking about your inability to dance or your filthy apartment, you know."

"Thanks, Connie," I said. "She's seen me without any clothes on, and then she still wanted to have coffee with me. I have to say I don't think I'm afraid of what else she might know about me." Of course, I really hoped that Kim did know all there was to know about me, and love me anyhow, or know how to fix me. But I wasn't going to admit that to Connie.

"Well, it's your funeral. Or insane asylum or whatever." Connie focused on her sandwich.

--

Kim and I got closer quite naturally, and we began sitting side by side in Social Dynamics. She did see my filthy apartment and we did go dancing, quite a few times. But then a paper or test would come up.

"Hey Nate, want to swing by my place for dinner? I'm cooking lasagna." She meant really cooking, cutting up ingredients and applying heat the old-fashioned way, not just entering selections in a menu. She was even dressed for the occasion. She was wearing an apron and was covered in flour. It was like a white version of the all-blue outfit I saw her in on the night we met.

"God, that sounds good, Kim, but there's this paper due Tuesday."

"Aww, damn," she said. "Well, more leftovers for me, I guess."

"Come over and let's work on the paper together."

"Oh, I can't do that, Nate, the tomatoes would go to waste. Do you know how hard it is to get real tomatoes? It ought to be a crime to waste them."

"They'll make more. Come on, let me see your paper, maybe I can help you with it."

"That's sweet, Nate, but I don't want to look at my paper! I want to eat lasagna. With you. Are you sure you can't come by?"

"Sorry, baby. I really need to get through another two thousand words tonight if I'm gonna have enough time to revise. Some of us aren't able to just wiggle our nose and read Hendrick's mind, you know?"

Her eyes went wide at the suggestion. "Me? I would never! That would be completely unethical. Besides, he's always focusing too hard on the brunette in the third row to think anything useful." She giggled. "You know the one? With the huge lips?"

I couldn't help but laugh. "All right, Kim. I'll see you in class, then." I disconnected, and hoped she really would pull it together before the semester ended.

--

"What's it like, Kim? Being a telepath?"

"It's lots of fun!" Giggle. "The first part, knowing how people feel, it's not as different as you might think. It's sort of like the women's intuition people always talk about, except a little better. Then there's when you do a deep scan. That's like going into someone's bedroom, seeing all the things around them that they take for granted. Thoughts half-finished because they've thought them a million times before. The things they value, scattered here and there because they didn't know you were going to be visiting."

"Did you ever try to lie, and hide your telepathy?"

"You remember those esper tests we took in first grade? The guy who did mine was so scary. I knew he was an esper, and I thought if I was an esper he was going to take me away with him and I'd never see my family again. So when he asked me, you know, How many fingers am I holding up, Can you tell me what I'm thinking, I told him I had no idea. They told me he himself was an esper but I didn't realize what that meant." We shared a laugh at her younger self's frightened misunderstanding. "But he didn't take me away, so I thought he believed me. And then the next week, I got a badge, and I was so proud! It was my very first piece of grown-up jewelry. I think I still have it at home somewhere."

"Have you looked at that analysis matrix for Dynamics yet?"

"Oh, Nate! You're the worst at pillow talk. Let's go back to talking about me, that was great!" Giggle.

--

When I got back my C-plus in Social Dynamics, I thought Hendricks was a jerk but concluded that it was probably a fair grade. It had been a hard class. The TA published some grade statistics -- I was barely over the median. Some overachiever had gotten an A and spoiled the curve for the rest of us. Damn, I needed a better grade than C-plus to pull up my GPA. Hopefully I'd do better in one of the other classes I was taking.

I went to Kim's to tell her the news. She wasn't there but her AI let me in. I ordered a beer from the automat and, once it synthesized, lay on her bed and sipped it. I wasn't thinking about anything in particular when I heard Kim come in the front door.

"Command: Check messages. Audio only." she said, walking over to the kitchen to get something for herself. The AI followed her with the sound, so I didn't hear anything. "Next," she said. "Blah, blah, blah.. next."

She walked into the bedroom where I was just as the next message started. She didn't notice me where I was, but I heard the message: "Congratulations, Kim, you made the Dean's List. Again. Be well. Bye."

"Dean's List?" I asked, maybe more forcefully than I should have. Kim spun around and grabbed at her chest. "Oh my God, Nate, don't do that! You scared the shit out of me!" She giggled again. "How nice of you to surprise me like this! Man, you would not believe --"

"Dean's List?" I asked again. "How did you make Dean's List? You're the worst student in the entire Dynamics class! I only got a C-plus! You couldn't have done better than me!"

The giggle was gone now. She was staring at me calmly, rationally. She ran her hand through her hair, sighed. "Actually, Nate, remember the student who got an A? That was me."

"Oh my God." I closed my eyes, leaned back against the pillows. "You cheated, didn't you? I can't believe it. How could you? Don't you understand what this means? If they find out --"

"I didn't cheat, Nate." She spoke quietly. "You can't just mind-read an entire paper. Do you think I want to spend any more time with that TA than I have to? He's full of himself. Hendricks is even more full of himself and he's a lecher too. I wrote those papers myself."

"You lied to me," I said, scrambling out of bed. "I can't believe this. I'm out of here." I left the beer and broke into a run. I got back to my apartment sweaty and out-of-breath. "Command: lock, all," I gasped. I collapsed into a chair at the kitchen table. It took me some time before my heart rate returned to normal, but then I put my head on the table, covered with my scalp with my hands and hoped for the world to end.

--

Kim contacted me that night through the comm system.

"Is it that frightening that someone could be smarter than you?" she asked.

"It isn't about that," I said, though secretly I was afraid it was. "You lied to me."

"Oh, this is about the fact that you thought I was dumb. And that's supposed to be my fault. The truth is that I really am embarassed about my grades. It's hard enough being a telepath, let alone being the top student at this school too. But that wasn't good enough -- not only am I supposed to tell you the truth, I have to make you think it too."

"Don't give me that," I snapped. "You acted a certain way, knowing what I'd think. You encouraged me to believe you were a -- a --"

"A bimbo?"

"Yeah. You can't claim any moral high ground just because you never said the words. You're a telepath, you knew what I'd think! And you said exactly what it took to make me think it!"

"What was I supposed to say, Nate? Hi, I'm Kim, and while you've been staring at my chest I've been thinking about the next four hundred words I have to write for my paper due Thursday? I'm not responsible for other people's misunderstandings. I don't go around correcting people all the time. If you really wanted to know something about me, you should have asked flat out!"

"Oh, so now I'm supposed to ask explicitly about everything? Is it true that you're in my Social Dynamics class, or did you sneak in? For that matter, do you even go to school here or do you just go to the parties?"

"Don't get snippy, Nate.."

"How should I get? Don't you think lying is different for telepaths? Don't you think that manipulating someone into a misunderstanding of the world is bad, no matter what words you use? How can you expect me to trust you when you've misled me all along?"

"What difference would it have made? Did you want me to double-check your work? Provide you with crib sheets? Drop my pencil if the answer to question one was true?"

"Is that what you think I'm like? That I'm really that shameless?"

"What I know," she said, "is that you care an awful lot about your grades. And that you like to feel more intelligent than other people. Regardless of what I did or did not do, we had a happy few months together. Isn't that enough?"

"It's over, Kim," I said, with a finality I didn't feel. I don't know if she could read my mind over the commnet, but her expression didn't change, and she sat quietly for a few moments.

"Nate," she began, "I've lived with the thoughts of strangers for 20 years. In all that time, I've found out that people lie. They lie all the time. They lie to themselves, loved ones, strangers. And I didn't even do that. I dodged one question about my grades and I giggled a lot. If that's too much dishonesty for you, heaven help you if you ever fall in love." She disconnected.

I don't know if she graduated, moved away, or simply avoided me. Maybe I guessed right and she never really went to school there after all. I didn't see her there ever again. Life goes on.

Kim was right. Any woman who loves me is going to know me well enough to lie to me. Isn't love about trusting the other person anyway?

Connie says I did the right thing, that it's impossible to know whether a telepath of that caliber is lying to you. Connie thinks a miserable truth is better than a happy fiction any day of the week, and that I managed to get the miserable truth. But sometimes, when the library is empty, I think back to Kim and wonder if maybe I'd have been better served by loving the blue-suited woman who knew me well enough to lie to me.

Posted Thu Jun 7 10:46:14 2007
esvn.el

esvn.el: a minor mode for editing Ethan's journal

My journal

I keep my journal in my home directory in ~/writing/journal. But I'm obsessed with keeping records and history. Every time I make another entry, I enclose it in start and end times, like this:

12:51 PM

I wanted to keep sleeping, but the sun was shining right into my eye, so I got up instead.

--12:53 PM

Sometimes when writing short stories in my journal, I copy and paste the thing five or six times, with each copy having an extra iteration of proofreading. This is obnoxious as well as foolish; if you want history, keep your stuff in a version control system.

But svn requires commit messages for every commit!

And I usually don't have anything worth saying about the commit -- I added another thought. Or completed an old thought. The changes don't merit description, and looking at the change is usually enough to explain the change itself. Besides, I write my journal in emacs, and I don't want to have to keep a terminal open to type "svn commit -m 'blah'" every now and then. No, what would be nice is if emacs would commit for me.

There's a solution!

I spent one Saturday writing this minor mode. Basically it only has one feature -- if you type C-c C-c it commits. It will even prompt you for a commit message or accept defaults. If given a prefix argument, it will "autocommit", taking a message from an appropriate default and committing without user intervention.

In the future, perhaps there will be other features, such as calling svn diff, but I wouldn't bet on it.

Isn't this a completely nonstandard and possibly dysfunctional way to use SVN or any other version control system?

Yes. Ohhhh, yes.

What's the low-down?

esvn.el is some 150 lines of emacs-lisp. Be warned; I don't actually know emacs-lisp, so there are probably a number of horrible, awful mistakes and indiscretions sprinkled liberally throughout. If you have any comments, feel free to email me.

Known bugs:

  • It won't commit directories which contain the files it wants to commit.

  • It rebinds a key starting with the prefix key C-c. Naughty, naughty!

    I don't care. If python-mode can do it, so can I.

  • It doesn't use define-minor-mode, but instead writes its own function.

    I did this because I wanted to use the modeline indicator "E", but when I did this and turned on esvn-mode, I was helpfully informed that "Esvn modE enabled".

Recent versions:

  • 0.1.3, 2007 April 29: Display only the last line of SVN output, and always save on commit.

  • 0.1.2, 2007 April 29: Might work with GNUmacs, but at least doesn't break on startup. Also, save before checking status.

  • 0.1.0, 2007 April 28: Does most of what I want it to do.

  • 0.0.1, 2007 April 28: An actual minor mode.

How do you use it?

I put esvn.el in my .xemacs directory and add the following to my .xemacs/init.el/.emacs:

(require 'esvn)

(define-derived-mode journal-mode text-mode "Journal"
  "Major mode for editing Ethan's diary."
  (setq esvn-default-commit-message "Entry.")
  (setq esvn-default-autocommit-message "autocommit")
  (setq esvn-default-add-message "New day.")
  (esvn-mode)
  (local-set-key [?\C-x ?\C-s] 'esvn-save-or-autocommit))

(setq auto-mode-alist
      (append
       '(
         ("\\.mdwn$"             . mdwn-mode) ; completely unrelated
         ("writing/journal/"     . journal-mode) ; see [1]
         ) auto-mode-alist))

Now, whenever I open a file in writing/journal/, emacs will fill in some default commit messages. Note also that I rebind C-x C-s to autocommit by default! Of course, if I want to provide a commit message, I can do so using C-c C-c.

This is how I use esvn.el. I have no idea how you might use esvn.el. It is possible that esvn.el might have some use besides editing my journal, but I doubt it.

[1] I know that this matches any writing/journal/ directory, including invisible_writing/journal/, etc., but there's only one directory called anything like that in my home directory, and when I was testing this it was useful to mess around in another directory (tests/writing/journal).

Why didn't you use emacs's built in VC-mode?

I looked at emacs's built in VC a couple of years ago and it didn't seem like what I wanted. Of course, I wasn't as expert in emacs-fu as I have become over the course of writing esvn.el, so I might have been able to hack VC up.

On the Saturday that I wrote esvn.el, I looked at emacs's VC again, and saw immediately that it supported "SCCS, RCS, and CVS", and was so horrified that I decided to ditch.

Why didn't you use psvn.el?

psvn is too heavyweight. psvn takes a project-based view to svn use -- you have a command that pulls up a magic svn buffer, which you can then use to mark up particular files for committing or not committing or adding or deleting. I wanted something where I could press a key and commit the file I was working on. psvn doesn't do that, it does something else. So I wrote my own.

What are the core assumptions of esvn.el?

  • Each file is committed "by itself", after possibly adding it to version control. No two files are committed together. In other words, changing one file is assumed not to necessitate changing another file.

  • It is assumed that no files are ever removed from version control, but files are added.

  • In my .emacs snippet, I assume that all files in a particular directory are going to be placed in version control.

  • It is assumed that changes are going to be frequent, small, and self-explanatory.

esvn.el is unlikely to be useful for any programming work, but for plaintext it may be handy.

Aren't you going to have a huge SVN repo with zillions of transactions?

The KDE SVN repo has over 260,000 revisions. I do like to whine at great length in my journal, but even so, I don't expect to have that many.

Where can I get it?

Here.

Posted Sun Apr 29 00:53:20 2007
puppetry index

I'm a big fan of the webcomic Puppetry, but even so I have to admit that flipping through her archive is mostly a hit-and-miss proposition. And whenever I think of a random strip, I can't do much better than a linear search of the whole archive. So here's an index of what I feel are the remarkable strips, intended to help me find the ones I'm looking for next time without paging through all the boring ones.

If you haven't read Puppetry before, go to the link above and give it a shot. This is like my personal "greatest hits" list, and should be considered spoilers.

2003:

2004:

2005:

2006:

2007:

2008:

Posted Thu Apr 5 09:59:46 2007
IDE Python Triple-Quote Shootout

It turns out that emacs's current python-mode (and xemacs too) doesn't handle certain triple-quoted strings properly. In fact, they don't handle them at all; they match single-quoted strings as follows:

"""Text"""
[][----][]

That is, the first pair of quotes is matched as an empty string, and the last pair is matched as an empty string, and anything in the middle is matched as in an ordinary string. Meaning that misparses can happen if you try:

"""This is a double quote: ". Still in the quote!"""
[][------------------------]                     [][

Of course, you can escape the quote, but that gets old fast:

"""This is a double quote: \". Still in the quote!"""
[][-----------------------------------------------][]

There is a patch in sourceforge. You might be overwhelmed by the complexity of the solution -- I was, and so I poked around the emacs documentation and came to the conclusion that the emacs syntax-highlighting engine is not designed for this sort of thing. The Emacs modify-syntax call appears to take a bunch of flags, which (taken together) allow comment delimiters of two characters but that's it. Every other syntax class begins and ends with a single character.

Spurred by a discussion on the Internet, I decided to see how some other editors fare on this problem.

First up is emacs/xemacs. (This is without python-mode from sourceforge.)

Both fail. (I think python-mode is shared across emacsen here.)

Next I decided to try the old rival, vim:

It passes.

I tried Kate:

KWrite is the same, only with a fixed-width font.

For fun I tried Python-specific IDEs Eric and Idle:

I don't use Eric much, or else I might go crazy -- man, what a busy interface!

Not surprisingly, both pass. I decided any Python-specific IDE probably would, so I switched back to generic IDEs.

Here's jed:

It fails. I decided to add another line of comment so it was clearer what went wrong. Note also the color of the period.

Here's something called fte that I turned up in aptitude:

It passes.

I finally figured out that an appropriate GNOME editor would be Gedit:

It also passes.

So, most editors with special Python modes handle this case correctly. Those that don't, handle it incorrectly in a particular way -- they fall back on single-quote-parsing to handle it, which means it can be interrupted by a single quote. Those editors are: jed and emacs.

Of course, this is a pretty minor acid-test for an editor. Unfortunately, I don't have the time to really use each of these editors and form a thorough, grounded opinion on them. Take this analysis, if you could even dignify it with that term, with a grain of salt, but enjoy the pretty pictures.

Posted Fri Mar 30 12:48:50 2007
yahoo towers

At a friend's recommendation I started playing Yahoo Towers. I've played a bit of Tetrinet in my day and the game seemed pretty similar, although obviously different in many ways. I've had some time to think over my impressions of the game and I've come to the conclusion that I like Tetrinet better. Maybe this will change as I get better at Yahoo Towers, but I doubt it. Here's my list of grievances.

Problems in this version

In general, the game is jumpy and disorienting.

Stones are sometimes hard to see; I've made plays which turned into misplaces because I didn't see a stone.

The backlog of powers is too small. You get to hold maybe 8. Maybe the intent is that you use all powers as soon as you get them, I'm not sure.

There's no way to throw away a power without using it.

Lag prevents you from moving, enabling you to only watch as the piece falls and destroys your plans or just kills you.

No record of power uses, so you can't scroll back to see "what killed me?"

Design flaws

The first and most important design flaw is that each move is too complicated. This makes it impossible to take your eyes off the screen long enough to choose who to fire a power at. Instead there's a "fire at random person" binding for the space bar. In order to support this, powers come in two flavors, "offensive" and "defensive". Offensive powers are fired at enemies, defensive powers only at yourself.

Because you can't choose who to shoot a power at (you can, but it's too difficult), frequently powers that are marked defensive get used on you at random that mess up your planning. (Midas Block, Color Plague, or Color Blast all do this.) Setting up an intricate combo or yahoo? Too bad! In Tetrinet your enemies can screw up your planning (blockquake!), but in Yahoo Towers so can your partner.

Crap can fall on you at any time, including during a play. A move that is a combo when the block is at the top of the screen becomes a misplace before it hits the bottom. Again, this makes it impossible to "set up" a move or at any time take your eyes off of your stack.

Stones are nearly impossible to get rid of. Tetrinet does this much better -- even a block bomb at a crucial moment only produces blocks that can be cleared in lines. It may be that having stones in your stack is not an unrecoverable handicap, but it sure seems that way to me now.

Powers can be removed from your depot before you play them.

Posted Tue Mar 27 13:34:04 2007
taped the internet

The anachronistic combination of line-art clipart style and VHS tapes with the Internet makes me smirk philosophically. The image itself isn't all that great, though.

Posted Sat Mar 10 21:49:34 2007
increase indent

Emacs doesn't have any keybinding to these functions by default ... nor menu-items, nor anything else. You might find them helpful when working with text formatting languages like Markdown or reST.

(global-set-key "\C-c>" 'increase-left-margin)
(global-set-key "\C-c<" 'decrease-left-margin)
(global-set-key "\C-c\C-r" 'increase-left-margin)
(global-set-key "\C-c\C-l" 'decrease-left-margin)

These keybindings come from emacs's python-mode. Note that the prefix key C-c is "reserved for users" -- nevertheless, python-mode sets 28 keybindings beginning with it.

Posted Sun Feb 25 20:45:16 2007
Last edited Sun Apr 29 00:53:20 2007

Blue Sky design by Jonas John.