Issue 2-49, December 10, 1997

Be Engineering Insights: Terminal Issues

By Rico Tudor

This article discusses certain design issues of Terminal, the terminal emulator for BeOS, and related matters.

GUI Versus Command Line

There are two ways to interact with BeOS. The first is apparent from the moment the system boots: Take the mouse and click on the pretty pictures. This allows the beginner to use the system immediately. For advanced users, however, the point-and-click routine becomes tedious, resulting in such "impure" features as keyboard accelerators.

The second way to interact with BeOS takes the keyboard accelerator concept to the logical conclusion: Dispense with the mouse, and use the keyboard only. This happens when you start the Be app called Terminal. It reads a line of text >from the keyboard, and interprets it according to well-defined rules.

Real-world users rely on a mix of these two approaches: The painting program needs a file name (which you type), while Terminal users can construct new command lines by cutting-and-pasting.

The Posix-compliance of BeOS is not merely a crutch to allow developers to port archaic programs. The persistence of UNIX-like systems, and the accompanying terminal emulators, shows the attraction and power of expressing your computational desires in a symbolic way. From a contributor to the "fortune" command:

Re graphics: A picture is worth 10K words—but only those to describe the picture. Hardly any sets of 10K words can be adequately described with pictures.

GUI fanatics are free to send me a picture of this quotation!

Plumbing

Terminal is mistakenly considered a monolithic application that reads and processes commands. It is actually one of several modular components, which can be recombined in useful ways. Of chief interest is the "shell", to use the UNIX terminology. The shell is a program which reads a command line, parses it, and then executes another program to do the work. The /bin directory contains many useful commands, familiar to UNIX users. The shell shipped by Be is "bash", but is easily replaced at the user's whimsy.

Here are some BeOS configurations:

bash <-> pty <-> Terminal <-> AppServer <-> keyboard/screen

bc <-> pty <-> Terminal <-> AppServer <-> keyboard/screen

bash <-> pty <-> NetServer <-> network card

bash <-> serial driver <-> serial port

The first configuration applies when you double-click the Terminal icon, or type "Terminal" to the shell. The second applies if you type "Terminal /bin/bc", which starts the "bc" desk calculator instead of a shell. A command other than "bc" may be specified, of course. The third applies if you connect to your BeOS machine from the network: You must enable "telnet" service in Network preferences. The last configuration will be supported in the next release, and permits remote logins from a serial device, such as a modem.

The pseudo-tty driver, "pty", serves the same purpose as that in the UNIX world. Both "pty" and the serial driver provide uniform services in the area of low-level character handling. This handling is programmable using the "termios" family of functions, described in any Posix manual. From the shell, use the "stty" command: Type "stty—help" for help.

Note that the shell is usable in a non-interactive setting, reading commands from a file. This scripting ability is enhanced by its built-in programming primitives. Look at /system/boot/Bootscript for an example. A "shell command file" can be started from the Tracker just like any Be app. The "bash" manual gives exhaustive details about shell programming.

Termcap And Curses

UNIX pioneered the notion of describing the characteristics and abilities of standalone terminals, a.k.a. glass TTYs. There was a plentiful assortment in the 70s and 80s, with differences that could best be described as gratuitous. The publicly assembled file, stored in /etc/termcap, allowed programs like text editors and games to move the cursor all over the screen, without knowing the particular brand of incompatible tty being used.

This file is shipped with the BeOS, and comes in handy when you want to run "vi" or "emacs" locally in a Terminal window. It is also needed if you log into a BeOS system from Solaris, or from that VAX VMS system with the TeleVideo.

The BeOS will also ship with "ncurses", a powerful library built on termcap, and used by many terminal-oriented programs. This is based on the "curses" library of BSD UNIX.

In the 90s, most terminal manufacturers have departed the business (or this world). The market remains for terminal emulators, such as ProComm, Kermit, xterm of X Windows, and all UNIX workstation consoles. Be's Terminal app falls in this category. All follow the loosely-defined ANSI X3.64 standard.

Character Sets

The character set used by Terminal is 8-bit. The appearance is determined by the font selected by the user—usually some approximation of ASCII, or ISO Latin1 for Europeans. The BeOS supports 16-bit Unicode, but the design impact on Terminal is undecided. The BeOS has selected the Unicode byte encoding called UTF-8. The ASCII character set is 7-bit, and maps one-to-one onto UTF-8. However, bytes in the range 0x80–0xFF start a multi-byte UTF representation of a Unicode character. This conflicts with 8-bit character sets, like Latin1.

This is only one example of the conflict between Terminal's mandate to faithfully emulate the past, and the desire to keep one foot in the present.

ANSI Compatibility

The ANSI standard defines some aspects of terminal behavior, such as the effects of various escape sequences. The standard is too vague to implement a working emulator; the successor standard, ISO 6429, is similarly deficient. For example, cursor behavior at end-of-line is undefined: A program cannot know whether to expect line truncation, line wrap, or a flashing blue screen.

Instead, writers of "ANSI" emulation must rely on existing TTYs, such as the old VT family from DEC, or other emulators. This is much better than the free-for-all in past decades, but is still aggravating for users.

The problem is not solved by termcap. Observe that you can use "telnet" in Terminal to log into a remote system. /etc/termcap on the BeOS is not used in this case. If the other system has termcap support, you can transfer the "ANSI BeBox Terminal" entry to the remote system. Otherwise, you must cross your fingers and claim to be "vt100", "linux", "xterm", or some such.

The best approach would be for Terminal to emulate several well-known commercial products, allowing the user to make whatever claim that works for the remote system. Needless to say, this is an arduous task.

Further complications arise in the keyboard department. The existing standards say nothing about keyboard layout, or the characters and escape sequences to be sent by those keys. The current situation is one of bewildering variety. Many emulators have programmable keys, but this only places the burden on the user.

The current goal is to have Terminal emulate all useful aspects of the DEC VT220. A large fraction of these capabilities have already been implemented.

Text Selection

Like every good Be app, Terminal provides access to the system-wide clipboard, allowing text to be copied and pasted. The usual menu method and keyboard accelerations are employed.

For users with a 2-button mouse, the rightmost button will paste text from the clipboard. As a further shortcut, if the window contains selected text, this text, rather than the clipboard, will be pasted.

Since the cursor can be moved anywhere on the screen, and overwrite any portion, the definition of the "correct text" to copy and paste is sometimes unknown. This is a problem for all terminals and emulators with text retrieval ability. The usual route taken is to copy the text as it appears on the screen, but that means tabs are replaced by spaces. It also creates trouble for multi-line copies, where text may have been wrapped.

Character Attributes

The appearance of characters can be embellished by the use of "attributes". Terminal, for example, allows underlining, reverse video, bold, background color, and foreground color. These attributes can be combined freely, using the appropriate escape sequences. The permitted colors are white, black, red, green, blue, yellow, cyan and magenta: watch out for unreadable combinations!

When you start a new Terminal, the "Be" logo is generated using various attributes: underline, bold, and color. Inspecting /etc/profile shows the trick. The following command can be typed to the shell verbatim:

echo -e "\nWelcome to the \033[1;4;34mB\033[1;4;31me\033[m shell.\n"

The [-e] flag to the "echo" command allows binary bytes to be represented with printable strings. So "\n" means ASCII Line Feed (octal 012), and "\033" means ASCII Escape (octal 033). To enable the bold attribute, try

echo -e "\033[1m"

Then also enable a blue foreground with

echo -e "\033[34m"

Disable everything with

echo -e "\033[m"

Here is the complete list currently supported by Terminal:

1  bold
4  underline
7  reverse video
30 black foreground
31 red foreground
32 green foreground
33 yellow foreground
34 blue foreground
35 magenta foreground
36 cyan foreground
37 white foreground
40 black background
41 red background
42 green background
43 yellow background
44 blue background
45 magenta background
46 cyan background
47 white background

Observe that no programming library like "curses", or any file like /etc/termcap, is needed to use escape sequences. You can use these embellishments in shell command files, or in C programs with printf(). In fact, all escape sequences are this simple to invoke.

Resources

For those interested in other escape sequences supported by Terminal, and terminals in general, the first file to peruse is /etc/termcap. A man page for "termcap" is needed to parse the entries, but a useful description of ANSI-style escape sequences is included near the end of the file.

Another excellent Internet resource is:

This contains a veritable trove of technical info, historical trivia, selected discussions from Internet news groups, and test programs.

Last, and least, you can order ISO 6429 from http://www.ansi.org/ for $130 or so. Save your money.


Developers' Workshop: Loading Add-ons from your Application's Directory

By Doug Wright

"Developers' Workshop" is a new weekly feature that provides answers to our developers' questions. Each week, a Be technical support or documentation professional will choose a question (or two) sent in by an actual developer and provide an answer.

We've created a new section on our website. Please send us your Newsletter topic suggestions by visiting the website at: http://www.be.com/developers/suggestion_box.html.

Wow, time flies when you're working at Be! Two weeks between these articles seems like mere minutes to me. So I had to take the scraper to the walls of my brain to get together something to tell you about. But first, this brief public service announcement...

We (Brian and I) have been working hard for the last couple of months to bring you a new, improved, more responsive DTS. If you tried it before and were disappointed with the response—or never got one—it's time to try it again. Come one, come all! Fill out the Developer Tech Support form in the Registered Developers Area. We are here to help. If we can't solve your problems, we'll hunt down your favorite Be engineers and pry the answers out of them.

DTS will become even more responsive as we move into 1998 with a new addition to the team. I'll tell you who in two minutes—I mean two weeks. Also, on the other side of the world, Be Europe has added a new developer support engineer, Thijs Stalenhoef. The combined efforts of our US and European DTS teams should serve you well as we move into the new era of Intel support.

Thank you for that DTS news update, and now back to our regularly scheduled program...

Speaking of lots to read, there is an incredible amount of information to absorb when learning to program for a new OS. A lot of the hard-to-get information about BeOS programming is hiding in back issues of the Newsletter. For example, check out these gems about add-ons from the not-too-distant past.

So you've read Cyril's articles about how to create add-ons, and where to put them depending on their use, and now you're actually going to try to load an add-on:

"I put my add-ons in a folder called Effects that lives in my app's directory. How do I load the add-ons from that folder?"

You can find the path to your app with...

status_t GetAppInfo(app_info *info) const;

app_info is a struct found in Roster.h...

struct app_info {
             app_info();
             ~app_info();

  thread_id  thread;
  team_id    team;
  port_id    port;
  uint32     flags;
  entry_ref  ref;
  char       signature[B_MIME_TYPE_LENGTH];
};

Use app_info.ref to find the path to your app and its parent:

app_info info;
BPath path;
be_app->GetAppInfo(&info);
BEntry entry(&info.ref);
entry.GetPath(&path);
path.GetParent(&path);

To add the Effects folder to the path use BPath::Append(). Append() takes a relative path string and adds it to the current path:

path.Append("Effects");

Now create a BDirectory using BPath::Path(). You can cruise through the directory and try to load each file:

BDirectory dir( path.Path() );

image_id  addonId;
status_t  err = B_OK;
Effect*   peffect = NULL;
BPoint    point(0,0), apoint;
BRect     rect;
Effect*  (*NewEffect)( char*, image_id );

//load all effects
while( err == B_NO_ERROR ){
  err = dir.GetNextEntry( (BEntry*)&entry, TRUE );
  if( entry.InitCheck() != B_NO_ERROR ){
    break;
  }
  if( entry.GetPath(&path) != B_NO_ERROR ){
    printf( "entry.GetPath failed\n" );
  }else{
    addonId = load_add_on( path.Path() );
    if( addonId < 0 ){
      printf( "load_add_on( %s ) failed\n", path.Path());
    }else{
      printf("load_add_on( %s ) success!\n", path.Path());
      if( get_image_symbol( addonId,
                           "NewEffect",
                            B_SYMBOL_TYPE_TEXT,
                            &NewEffect) ){
        printf( "get_image_symbol( NewEffect ) failed\n");
        unload_add_on( addonId );
      }else{
        peffect = (*NewEffect)( addonId );
        if( !peffect ){
          printf( "failed to create new effect\n" );
        }else{
          peffect->Init( this, point );
          peffect->GetButtonSize( (BPoint*) &apoint );
          point.y += apoint.y;
        }
      }
    }
  }
}

LoadAddon is a new sample app that covers all your loading add-on questions. You can find it at:

ftp://ftp.be.com/pub/samples/preview/add-ons/LoadAddon.zip

See you next time, and as always, if you need more info, don't hesitate to ask!


The Trouble With Choice

By Jean-Louis Gassée

This Monday's December 8th New York Times had a story in its Business section bemoaning the trouble with competing Web browsers. It reminded me of a conversation I once had, about 15 years ago, with a dinner guest at our house back in France...

We are sitting on the sofa, ready to watch the 8 p.m. newscast, uninterrupted by commercials in those blessed days. As a result, we get a fairly heavy dose of TV ads right before the male or female anchor starts telling us what to think.

On the screen, a Pampers commercial. The guest mutters words to the effect it's awful, I mechanically acquiesce. He keeps going and I suddenly realize he isn't protesting the aesthetics of the ad. What's the trouble? There too many brands of diapers, he grumbles. By now, I'm fully awakened from my TV stupor. If that's the case, who will decide how many and which brands? Us, he replies...

I forgot to mention the distinguished guest was both a psychoanalyst and a communist, and still is, I'm afraid. By us, he meant the Party or some Committee or Directorate in charge of diapers. The discussion didn't go very well.

Fifteen years later, reading the NYT story didn't go very well either. In essence, the writer described the trouble besetting us poor Web users because two browsers were not exactly compatible. Some sites even had the nerve of demanding you download IE 4.0 before being allowed to enjoy the riches they contain. Thankfully, the story stated this happened only in a small minority of cases.

Well, yes, it's a problem when the browser and the site don't agree completely, or at all. But who's going to decide how many browser we have, and which ones? Do we have too many browsers, or too few? To which the sophisticated reader might reply this situation arises because we don't have a Web standard or, if we have one, because the big players, Microsoft and Netscape, don't give a damn and are trying to increase their profits by generating a standard they'd control to their advantage.

There are many reasons not to let the apparatchiki decide for us how many and which Web browsers we should use. The Web is too young, the technology, the psychology and the sociology involved are far from being understood well enough. We've seen what happens when politicians try to legislate the contents of the Web. Do we want the same people who vote the Pentagon budget to also legislate HTML dialects? Why not microprocessor instruction sets? We've seen what happened with a programming language, Ada, sanctioned by the DoD: No one uses it outside of mandated applications.

This isn't an argument about standards, international or not. Who can claim we don't need standards? But beware of the questions brooking no disagreement. If there are only pros and no con, then it's a con. One Web standard would make life much simpler, wouldn't it? Perhaps—as long as standardization isn't premature, doesn't brutally stop the innovative fermentation taking place while the technology is still young, or doesn't allow anyone to own the standard and to create a profitable monopoly at our expense.

Two browsers—only—is definitely a bad sign. Something or someone has caused the Web to age prematurely and we should treasure the remaining vitality still manifesting itself in experiments—and in a little bit of confusion.

Of course, this assumes there is a "free market" at work for the Web standards decisions, as in earlier programming language contentions. This assumption is a weakness in my argument, one that might unfortunately cause government bureaucrats to offer their help in repairing the broken free-market process.

Creative Commons License
Legal Notice
This work is licensed under a Creative Commons Attribution-Non commercial-No Derivative Works 3.0 License.