Taiga Nakamura

pdiff.py: side-by-side graphical diff as HTML

2007-02-22T22:36:24Z

pdiff.py is a super-simple Python script to produce a side-by-side graphical diff as HTML. It merely calls the difflib library, and send the output to the standard output. Python 2.4 or newer is required. Note that I have written this script as a part of a larger application. If you are just looking for a pretty graphical diff application, I'm sure you will find a better one at somewhere else :)

CentOS 4.2/4.3/4.4 on ThinkPad T43

2006-12-14T22:00:49Z

After everything I've gone through with the "breeding-edge" Fedora on my old ThinkPad T40 (read here and here), I decided to go with something a little more "stable" for a new laptp, ThinkPad T43. (No offense to Fedora - I still love it in many ways.) What I chose was CentOS. CentOS 4, one of the popular "Red Hat clone" distributions, is basically a free recompilation of Red Hat Enterprise Linux (RHEL) 4. The structure of the distribution is more or less similar to Fedora Core 3, as RHEL 4 was being developed in parallel with FC3. (Therefore, there is a good chance that an RPM package for FC3 is compatible with CentOS, which comes handy.) Below is my setup memo. I hope this to be useful for others, but I make no guarantee of any kind whatsoever. Please do everything at your own risk. Unlike the previous models of ThinkPad in the past few years, T43's partition for the system recovery doesn't seem to be "hidden". I have no idea what happened, but apparently they (I mean, IBM or Levono) went back to the method they were originally using in older machines; the system recovery partition appears to be a FAT partition at the end of the disk. If you want to do dual-boot, I recommend to be careful not to delete it until you first make a backup to DVD or CD media.

TODO list

Suspend/Hibernation
On-screen display
Modem
Bluetooth
Fingerprint detection

BIOS

I did nothing special in the BIOS setting specifically for Linux, except I disabled screen expansion in an attempt to use an external display for doing a presentation, but it didn't seem to mean much anyway (see here.)

Installation

Instal disk image for CentOS 4.2 consists of four CDs or one DVD. I ended up burning three CDs, as I didn't need to install anything in the fourth CD initially. You need a BitTorrent client to download a DVD image. Installation was very easy. I did custom installation with graphical mode, manually created partitions, and selected packages to install. I chose to disable SELinux. I chose "general LCD 1400x1050." After reboot, I successfully got a graphical login screen. Most devices (including trackpoint/UltraNav, USB, ethernet, audio, and volume buttons) worked out of the box. The only mystery was that my IDE disk (Toshiba MK1032GA) was recognized as a scsi drive (/dev/sda). The first thing I did after installation was to run "yum update" as root to install the update modules. You could also use 'up2date' or 'apt-get' for this purpose. It is also easy to add third-party repositories so that you can install the packages not included in the official CentOS repository through these tools. For example, you can add a file named /etc/yum.repos.d/*.repo to make yum access a third-party repository. (You may also have to do 'rpm --import' to import a GPG key.)

KDE

The default desktop environment of CentOS 4.2 is GNOME. I changed it to KDE by rewriting /etc/sysconfig/desktop to DESKTOP="KDE". This is just a matter of preference.

Wireless LAN

My T43 comes with the Intel PRO/Wireless 2200BG adapter. Although the CentOS kernel is built with ipw2200 and ieee80211 driver modules, wireless doesn't work after initial installation. You need to install the "firmware" provided by Intel. The easiest way is to download and install the firmware RPM available here (choose the latest package for Red Hat Enterprise Linux 4 i386). Alternatively, you can download the files from the SourceForge site and put them in /lib/firmware (The syslog in /var/log/messages indicates that CentOS 4 kernel needs ~~ipw-2.2-boot.fw~~ ipw-2.4-boot.fw. Newer kernels seem to need the version 2.4 of the firmware). After that, either reboot or run the following commands.

# /sbin/rmmod ipw2200
# /sbin/modprobe ipw2200

Note: kernel-2.6.9-42.0.2.EL had a problem with ipw2200 driver. ~~I'm forced to stay with kernel-2.6.9-34.0.2.EL until this is resolved. It also prevents me from upgrading to CentOS 4.4~~:( If you're interested in this problem, you might want to watch this bug in the Red Hat Bugzilla.) 10/08/06: The issue has been fixed in kernel-2.6.9-42.0.3.EL. You can check syslog to see if the driver is loaded correctly. Once this is done, you should be able to setup the wireless connection manually, with standard tools like /sbin/iwconfig etc., or a graphical configuration tool (/usr/sbin/system-config-network). More detailed and useful setup information can be found the project page. For example, the document says you may need to do:

# echo 100 > /sys/class/firmware/timeout

to load the firmware correctly (In my case, I didn't have to do this). It also provides newer versions of driver/firmware than what CentOS 4.2 kernel contains. If you have some problem, you may want to delete existing modules and install newer ones. As far as I believe, the "firmware" has nothing to do with what's written into the hardware EPROM, so testing a newer firmware doesn't affect anything when you are running a different kernel or a different OS.

Trackpoint scroll

While the trackpoint/touchpad works right after installation, there are several features missing. One of them is the "scrolling using the trackpoint", which allows the trackpoint to be used like a scroll wheel when the center button pressed. Since I heavily depend on this functionality, it was important to make it work. As far as I know, there are three possible approaches. The first one, which is simplest, is just changing the "InputDevice" section in /etc/X11/xorg.conf as follows.

Section "InputDevice"
        Identifier  "Mouse0"
        Driver      "mouse"
        Option      "Protocol" "IMPS/2"
        Option      "Device" "/dev/input/mice"
        Option      "ZAxisMapping" "4 5"
        Option      "Emulate3Buttons" "yes"
        Option      "EmulateWheel" "true"
        Option      "EmulateWheelButton" "2"
        Option      "EmulateWheelInertia" "50"
EndSection

After restarting X (logout and login again, or just reboot), the trackpoint scroll should work. Unfortunately, this isn't the perfect solution. It does make the scrolling using the center button work, but the center button no longer works for clicking. (That's why "Emulate3Buttons" is necessary - you need to press left and right buttons simultaneously to paste on X.) The second approach is to use tp-scroll (see here for the detail of how to set it up.). I have used it for Fedora for a while, but I haven't tried it on CentOS either. Basically it uses a FIFO created as a dummy mouse device, and the tp-scroll program, which works as a proxy to the real mouse device, tranforms mouse "signals" to realize trackpoint scrolling. It's a great program. The third approach, which I chose to take, is to use this patch, and recompile X.org from the source code. Then the following setting in /etc/X11/xorg.conf works (notice the new option "EmulateWheelClickToo").

Section "InputDevice"
        Identifier  "Mouse0"
        Driver      "mouse"
        Option      "Protocol" "IMPS/2"
        Option      "Device" "/dev/input/mice"
        Option      "ZAxisMapping" "4 5"
        Option      "Emulate3Buttons" "no"
        Option      "EmulateWheel" "true"
        Option      "EmulateWheelButton" "2"
        Option      "EmulateWheelInertia" "50"
        Option      "EmulateWheelClickToo" "true"
EndSection

Here's how to recompile the X.org RPMs from the SRPM package and the patch. Since I didn't want to "contaminate" the system directories for this task, I did it as a non-root user.

Download the following files: xorg-x11 SRPM, patch for trackpoint scrolling, and patch to the spec file. (I have adjusted the original patch so that it fits the source code of the CentOS.)
- Update for CentOS 4.3: the latest xorg-x11 SRPM for CentOS 4.3 is here. The patch for trackpoing scrolling and the pacth for the spec file still work, but since there has been slight changes in the line number in the latter, I put the new patch for the spec file here.
Set up the RPM build environment. I assume you build everything in the ~/rpmbuild directory as a non-root user. (The technique to use rpmbuild as a non-root user was borrowed from here.)
```
$ cd ~
$ cp -a /usr/src/redhat/ rpmbuild
$ echo '%_topdir %(echo $HOME)/rpmbuild' >> .rpmmacros
$ echo '%debug_package %{nil}' >> .rpmmacros
```
Install the SRPM file into ~/rpmbuild, and copy the patch file to the source/spec directory

$ rpm -ivh xorg-x11-6.8.2-1.EL.13.20.src.rpm
$ cp xorg-x11-6.8.2-emulate-wheel-click-too.patch ~/rpmbuild/SOURCES/
$ cp xorg-x11.spec.patch ~/rpmbuild/SPECS/

Patch the spec file and build the RPM

$ cd ~/rpmbuild/SPECS/
$ patch < xorg-x11*.spec.patch
$ rpmbuild -bb xorg-x11.spec

If everything goes well, the RPM files are created in the ~/rpmbuild/RPMS/i386 directory. Install them as root.
```
$ su
# rpm -Uvh --force *.rpm
```
After restarting X, you should be able to use both clicking and trackpoint scrolling with the "EmulateWheelClickToo" option. Delete the build file and directory (you could save the RPM files for later if you want, but you can always rebuild them.)
```
$ cd ~/
$ rm -R rpmbuild
$ rm .rpmmacros
```

Emacs

For certain reasons, I'm using the latest version of emacs on the CVS. Due to the exec-shield of recent Linux kernels, you need to use 'setarch' to avoid crashing during make.

$ export CVS_RSH="ssh"
$ cvs -z3 -d:pserver:anonymous@cvs.savannah.gnu.org:/sources/emacs co emacs
$ cd emacs
$ setarch i386 ./configure
$ setarch i386 make bootstrap
$ su
# setarch i386 make install

OpenOffice.org 2

CentOS 4.2 comes with OpenOffice.org 1.1.2. Unfortunately, OpenOffice.org 1.1.x has many limitations, and it's almost unusable for me. (The biggest problem is its inability to open a large spreadsheet.) I have used OpenOffice.org 2 and knew it's much better, but with the nature of CentOS (i.e., that of RHEL), it is unlikely that major update release is provided. Therefore, I obtained the RPM package from the OpenOffice.org project page and installed it. Since their package is for general linux environment, I had to take the following steps to fit it to CentOS. The version I installed is 2.0.1.

Uninstall OpenOffice.org 1.1.x. (It turns out that version 1.1.x and 2.0.1 can co-exist, as they are installed in completely different places. However, you may have to configure which application is invoked in the desktop setting. I just decided to uninstalled the old version completely, as I knew I never use it.) The following command:
```
$ rpm -qa | grep openoffice
```
will display the currently installed rpm packages for OpenOffice. You may want to take a memo so that you can remember which packages were installed in case you really need to reinstall them. As a root, run:
```
# rpm -ev openoffice.org openoffice.org-i18n ...
```
where "..." are the all relevant packages names. (The list depends on your environment.) In my case, it was
```
# rpm -ev openoffice.org openoffice.org-i18n openoffice.org-kde openoffice.org-libs openoffice.org-pyuno
```
If you have used OpenOffice.org 1.1.x, you may also want to delete the files in your home directory, such as ~/.openoffice-install-log, ~/.openoffice-lock, etc. manually. If you don't have to keep the setting of the old version, you can also delete the ~/.rhopenoffice1.1 directory. Otherwise, you may wait to delete this directory until you transfer the setting (see below). Logout and re-login so that the desktop icones for the old version disappears.
Go to http://www.openoffice.org/ and download the latest version of the package file (in my case, OOo_2.0.1_LinuxIntel_install.tar.gz). Extract the files, and install the RPM files as follows.
```
$ tar zxvf /opt/build/openoffice/OOo_2.0.1_LinuxIntel_install.tar.gz
$ cd OOA680_m1_native_packed-1_en-US.8990/RPMS/
$ su
# rpm -ivh *.rpm desktop-integration/openoffice.org-redhat-menus-2.0.1-1.noarch.rpm
```
You may want to select the RPM files to install if you don't need all functions. (I don't know enough about the dependency among those RPM files.)
Logout and login again to enable the updated desktop application information. (The 'kde-config' command may be useful if you have a problem with it.) In my case, I further had to manually change the KDE desktop settings: (1) Set the icon shown in Konqueror. Open the Control Center and go to "KDE Components" -> "File Associations". From the list, choose "application" and click "msword", "msexcel", and "mspowerpoint". Click the icon button and choose something like "openofficeorg-20-*". (2) Add the application launch buttons by right-clicking on the task Panel. (I'm not sure if you need similar settings in Gnome too).
Start the application from the menu. Alternatively, if you haven't configured the desktop, the OpenOffice.org 2.0.1 is installed under the /opt/openoffice.org2.0 directory, so run:
```
$ /opt/openoffice.org2.0/program/soffice
```
If the installations is successful, the setup window is displayed. If you accept the license agreement, you'll be asked whether you want to import the settings from the old version, and whether you want to register. Now you should be able to use OpenOffice.
If some Microsoft Office documents are displayed uglily, the cause is most likely a font problem. Since TrueType fonts such as Arial, Times New Roman, etc. do not exist in CentOS, the documents using these fonts can be shown with incompatible fonts. You can go to "Tool" -> "Options" -> "OpenOffice.org" -> "Fonts" to manually define font replacement table, but unless you have appropriate fonts it's not very helpful. (I remember the CentOS version of OpenOffice.org 1.1.x was showing these fonts with better look, but I wasn't able to look into why anymore as I had uninstalled it already.) A solution is to install "alternative fonts". There are many freely available fonts you can find on the internet (see here for example). Put the extracted font files in /usr/share/fonts (you can create any sub-directories), restart the font server and re-login (or just reboot the system).

IBM ThinkPad ACPI Extras Driver

I have installed the ibm-acpi driver for ThinkPad ACPI. The ibm-acpi driver is now part of kernel 2.6.10 and later, but as of today, the version of CentOS kernel (i.e., RHEL 4 kernel) is still 2.6.9-x. Among other things, I needed to install this driver to be able to disable screen expansion when doing a presentation (see here). I expect It will be useful to setup suspend/hibernation, although I haven't tried it yet. Since the RPMs found in several sites were either too old or seemed incompatible with Red Hat-based distributions, I just downloaded the source code from the project page and compiled it according to the README document. (This must be repeated when you update the kernel.)

$ tar xzvf ibm-acpi-x.y.tar.gz; cd ibm-acpi-x.y
$ make
$ su
# export PATH=$PATH:/sbin
# make install

Then run '/sbin/modprobe ibm-acpi' and you will be able to access the driver via /proc/acpi/ibm. The final step is to load the driver module automatically. I added the following line at the end of /etc/rc.d/rc.local.

/sbin/modprobe ibm-acpi

Of course you could run modprobe by hand every time you boot the OS, or if you want to do something more sophisticated, it would be easy to write an init.d script to make modprobe/rmmod easier. The version of KDE (3.3.x) included in CentOS has the "IBM Thinkpad Laptop" menu in the Control Center, but unfortunately it doesn't seem to work with this ibm-acpi driver.

Doing a presentation

Sometimes I do a presentation with an external projector. The LCD of my T43 is SXGA+ (1400x1050 pixels), but no projector that I can use supports such a high resolution. An ideal situation would be to show low-resolution image (like XGA, 1024x768) while keeping SXGA+ display on the LCD, but I have had no clue how to do it. The more practical way is to switch to a low LCD resolution when doing a presentation. KDE can change the display resolution in a "Window-ish" way (right-clicking on the desktop and choosing the "Configure desktop" menu), and this is acceptable for me. However, there's still a problem. Even if I reduce the LCD resolution to XGA, the external RGB output is still corrupted if the LCD screen expansion is enabled. Worse yet, even if I disabled the screen expansion on BIOS (and the config tool on Windows XP), the screen expansion is still enabled on CentOS. I haven't figured out how to turn of on X (or KDE, I don't know). The only solution I've had so far is to rely on the ibm-acpi interface, and manually disable the screen expansion when necessary. (I don't know how to obtain the current screen expansion mode.)

$ echo expand_toggle > /proc/acpi/ibm/video

I hope to automate this (or a cleaner solution) and make it work through the Fn+F7 key in the future.

Software Mixing

Traditionally, Linux applications directly access the sound device (/dev/dsp), which means only one application can play the sound at one time. While this may be good for high-fidelity playbacks, I really wanted to play some music, listen to the sounds from Flash on Firefox, and hear the notification from KDE all at the same time. To do this, we have to use a mechanism for software mixing, and make applications go through a "sound server" instead of accessing the bare sound device. There are several ways to achieve this. Some sound servers are closely coupled with desktop environments like KDE or GNOME, while others are independent of them. If you are interested, try googling arts, esd, jack, sdl, etc. The approach I chose to take was dmix, which is a native plugin for ALSA (Advanced Linux Sound Architecture), because it was easy to use, and it seemed to cause fewest side effects. This page contains the overview of the necessary settings, and here's what I have actually done. It's mostly based on this page (written in japanese).

Save this file as /etc/asound.conf. This file is read by /etc/alsa/alsa.conf. If you want to limit the configuration to one user, it is possible to put this file as ~/.asoundrc instead of /etc/asound.conf.
Install alsa-oss to use the aoss command. Aoss is a wrapper script to allow non-supported applications to force using ALSA. You can go to here, download the latest release and install it. Instead, I downloaded the SRPM package from here and compiled it.
```
$ cd ~
$ cp -a /usr/src/redhat/ rpmbuild
$ echo '%_topdir %(echo $HOME)/rpmbuild' >> .rpmmacros
$ echo '%debug_package %{nil}' >> .rpmmacros
$ rpm -ivh alsa-oss-1.0.10-1.src.rpm
$ cd rpmbuild/SPECS
$ rpmbuild -bb alsa-oss.spec
$ cd ~/rpmbuild/RPMS/i386
$ su
# install -ivh alsa-oss-1.0.10-1.i386.rpm
```
If you can find a alsa-oss RPM package for RHEL4 or Fedora3 somewhere else, it will probably work too.
Change application configurations to use ALSA.
- KDE: In the Control Center, go to "Sound & Multimedia"->"Sound System"->"Hardware" and select "Advanced Linux Sound Architecture".
- MPlayer: create a file ~/.mplayer/config and add 'ao=alsa'.
- XMMS: open the preferences window and choose ALSA as Output Plugin.
- Firefox: create a wrapper script.
```
#!
aoss /usr/bin/firefox "$@"
```
  Then make it executable and put it in a location listed in the execution path. For example, you can put it in the ~/bin directory and add this directory to the $PATH environment variable. (Another approach is to create an alias like "alias firefox='aoss /usr/bin/firefox'".)
- Play: add "alias play='aplay'".

Other minor issues

The default paper size of dvips is A4. US users may want to change it letter (see here).
To use tpb for onscreen display, you need to install xosd RPM. Old version of xosd had an issue when it is used together with VMWare (tpb randomly freezed when VMWare or VMPlayer was running). xosd-2.2.14 or later works fine. (You may have to create a device file /etc/udev/devices/nvram manually if your RPM file doesn't include it. The RPM package for RHEL4 does contain the device file. try 'rpm -qf tpb' and see what files are included.) Finally, to run tpb as a normal user, you need to edit /etc/udev/permissions.d/50-udev.permissions and edit the line for nvram to something like 'nvram:root:root:0666'. You could directly changing the permission of /dev/nvram, but then that's only effective until you reboot the system.

Upgrading from CentOS 4.2 to 4.3

With the release of the Red Hat Enterprise Linux 4 Update 3, the CentOS 4.3 has been made available. On my system, the yum command automatically attempted to connect a new repository. (To prevent this from happening, you can modify /etc/yum.conf and the files in /etc/yum.repos.d so that yum can access to the version 4.2 tree instead of the latest version 4 tree.) It is often unadvised to jump to a new release if the stability has the highest priority. However, I took a risk and just ran 'yum upgrade'. After a reboot, the system seems to work fine as before. (Since the X packages are updated, you need to rebuild the X packages to re-enable the center-button click if you are using the same approach as I do.)

Upgrading from CentOS 4.3 to 4.4

Unless you configure not to, the "yum update" command makes the system upgraded to 4.4. I recommend to use "yum upgrade" to check obsolete packages. After upgrading is complete, you have an option to run "yum clean all" to free up the disk space for the downloaded packages.

Collection of Software Defects in General Domains

2006-12-04T19:07:22Z

There are many anecdotes about software defects that led to major incidents. Here I maintain a list of websites about such "famous" defects. * Collection of Software Bugs by Dr. Thomas Huckle * A Collection of Well-Known Software Failures by Dr. Gang Tan

HPCBugBase: An Experience Base for HPC Defects (Poster)

2006-11-15T05:00:00Z

Taiga Nakamura, "HPCBugBase: An Experience Base for HPC Defects", poster presented at the International Conference for High Performance Computing, Networking, Storage and Analysis (SC06), November 11-17, Tampa, Florida, 2006. Awarded the third place in the ACM Student Research Competition. Abstract: We present the design and implementation of HPCBugBase, an experience base for high performance computing (HPC) software defects. Our goal is to accumulate empirical knowledge about commonly occurring defects in HPC codes using an incremental approach. This knowledge is structured so that HPC practitioners such as programmers and tool builders can use it to reduce debugging costs, as well as provide feedback which becomes incorporated into the system. By building the experience base, we expect to help the process of making explicit the knowledge about recurring defects that otherwise cannot be shared. The current system is built on a Wiki system, which allows incremental accumulation of data at various levels of abstraction. We implement additional analysis functions that do not exist in a generic Wiki system as custom plug-ins. We have populated the system with data collected from software engineering studies from the DARPA High Productivity Computer Systems Project.

Experiments to Understand HPC Time to Development

2006-11-01T05:01:00Z

Hochstein, L., Nakamura, T., Basili, V. R., Asgari, S., Zelkowitz, M. V., Hollingsworth, J. K., Shull, F., Carver, J., Voelp, M., Zazworka, N., Johnson, P. "Experiments to Understand HPC Time to Development ," CTWatch Quarterly, Volume 2, Number 4A, November 2006. [PDF] http://www.ctwatch.org/quarterly/articles/2006/11/experiments-to-understand-hpc-time-to-development/

Observations about Software Development for High End Computing

2006-11-01T05:00:01Z

Carver, J. C., Hochstein, L. M., Kendall, R. P., Nakamura, T., Zelkowitz, M. V., Basili, V. R., Post, D. E. "Observations about Software Development for High End Computing," CTWatch Quarterly, Volume 2, Number 4A, November 2006. [PDF] http://www.ctwatch.org/quarterly/articles/2006/11/observations-about-software-development-for-high-end-computing/

What's Working in HPC: Investigating HPC User Behavior and Productivity

2006-11-01T05:00:00Z

Wolter, N., McCracken, M. O., Snavely, A., Hochstein, L., Nakamura, T., Basili, V. "What's Working in HPC: Investigating HPC User Behavior and Productivity," CTWatch Quarterly, Volume 2, Number 4A, November 2006. [PDF] http://www.ctwatch.org/quarterly/articles/2006/11/whats-working-in-hpc-investigating-hpc-user-behavior-and-productivity/

Survey of Empirical Research on Model Driven Approaches

2006-10-16T03:20:59Z

Model Driven Architecture (MDA) is a relatively new software design approach proposed by OMG. I was curious how much empirical research has been done on MDA or related topics, so I have done some survey. I have identified only a few so far, but I will update the list as I find more references. B. Anda, K. Hansen, "A Case Study on the Application of UML in Legacy Development", ISESE'06, 2006.

"those who applied UML in modelling and enhancing legacy software experienced more challenges and fewer benefits from the use of UML than did the developers who modelled and developed new software"

Several experience reports have been presented at the MoDELS 2005 Conference. The OOIS-MDSD Workshop mentioned the following topics in the call for participation: * Empirical validation of the value of modeling, especially using the OMG's UML and MDA * Empirical support for the effectiveness of model-driven and agile development approaches * Empirical validation of the benefits of Agile development (XP, Scrum, etc.) However, the actual program didn't seem to cover empirical results.

Poster at SC06

2006-10-08T00:39:04Z

I will present a poster at the SC06 conference. Here is the information. Update: This paper has won the third place in the ACM Student Research Competition.

Identifying Domain-specific Defect Classes Using Inspections and Change History

2006-09-22T05:00:00Z

Taiga Nakamura, Lorin Hochstein, Victor R. Basili, "Identifying Domain-Specific Defect Classes Using Inspections and Change History", Proceeding of 5th ACM-IEEE International Symposium on Empirical Software Engineering (ISESE'06), September 21-22, 2006, Rio de Janeiro, Brazil. [PDF] Abstract: We present an iterative, reading-based methodology for analyzing defects in source code when change history is available. Our bottom-up approach can be applied to build knowledge of recurring defects in a specific domain, even if other sources of defect data such as defect reports and change requests are unavailable, incomplete or at the wrong level of abstraction for the purposes of the defect analysis. After defining the methodology, we present the results of an empirical study where our method was applied to analyze defects in parallel programs which use the MPI (Message Passing Interface) library to express parallelism. This library is often used in the domain of high performance computing, where there is much discussion but little empirical data about the frequency and severity of defect types. Preliminary results indicate the methodology is feasible and can provide insights into the nature of real defects. We present the results, derived hypothesis, and lessons learned. http://doi.acm.org/10.1145/1159733.1159785

UMDInst: Instrumentation Tool for Software Engineering Experiments

2006-08-31T03:30:42Z

UMDInst is an instrumentation tool for software engineering experiments developed by the ESEG group. It was originally written by Lorin Hochstein, and I am the current lead developer. It's available at the UMDInst page. Unlike many other instrumentation tools designed to collect data in a specific IDE (such as Eclipse, Visual Studio), UMDInst is designed to collect data in a remote login environment. A particular use of this tool is in the high performance computing domain. Users of supercomputers usually develop, debug and run the program on a remote shell, making use of various command-line tools and GUI-based tools. There is often a job scheduling queue which the users must interact with to get their program to run. Since the type of tools being used and the underlying system configurations are highly heterogeneous, existing instrumentation tools cannot be applied to collect data in this environment. UMDInst takes a "wrapper approach." When installed, it generates a set of wrapper scripts which are called instead of the real tools (compilers/debuggers/profilers). A wrapper then calls the corresponding real tool, and acts as a transparent proxy to capture necessary information. The main component of UMDInst is written in Python. Currently, Python 2.1 or later is supported. UMDInst also supports Hackystat as a supplementary data source. Java 1.5 or later is required to use Hackystat.

HPCS Project Website @ UMD

2006-06-23T00:52:24Z

The HPCS project website at UMD is now http://care.cs.umd.edu:8080/hpcs/. Since we mainly represent the Development Time Working Group, this is also an official site for this WG.

Empirical Paradigm in Software Engineering

2006-01-20T20:16:08Z

One of the most important insights gained from more than 30 years of history in software engineering history is that one solution never fits all: no single "silver-bullet" model, practice or technology works in all contexts. Appropriate solutions depend on the project goals, requirements, application domain and other factors, and it is common that a solution that worked wonderfully in one project results in a catastrophic failure in others. Therefore, an important question is "what works when, and why." A factor that makes it difficult to answer this question is that software development is a human process. Due to the complexity and uncertainty of human activities, reasoning from theory is often insufficient to understand true implications. The above discussion leads to a need of empirical research in software engineering. An empirical paradigm involves the following steps: * Building models * Experimenting * Learning, encupsulating knowledge Note that this involves an iterative improvement.

OTJava: Ownership Type System for Java

2005-12-30T22:45:36Z

OTJava is an implementation of Ownership Type System for Java, which I developed as a class project. It works as an Eclpse plugin. The type system is based on David G. Clarke's paper in 1998. Usage To use the ownership type system on OTJava, a user must write ownership annotations in the source code. In the current version, annotation must be written using javadoc-style comments. Each annotation should be place right after the type. Since the original type system doesn't fully cover the entire Java language, neither does this tool. In fact, it doesn't even support inheritance. There are more sophisticated ownership type systems developed by Klarke and other researchers, so it would be desirable that the future version of this tool will use them. The primitive types (int, double, ...) cannot have context parameters. They can only have an ownership parameter. Implementation The implementation depends on the abstract syntax tree provided by the Eclipse JDT (Java Development Tools) library. The library is very useful as it can not only parse the source code and build a syntax tree, but also provide name resolution and type binding. On the other hand, it could consume huge resources especially when analyzing a large program. Therefore, care must be taken to avoid an out-of-memory error. The current version traverses the ASTs for given source files exactly twice. When the number of files are small, the tool can save all ASTs for all classes. Since this approach requires a lot of memory for a large program,

In the first path, the tool builds a table of classes which contain an ownership annotation. For each class in the source files to be analyzed, register the ownership types of fields and methods. The classes not analyzed in this step (including library classes) are considered to have no ownership annotation. Each annotation in field and method declarations is checked against
In the second path, the tool inspects the actual content of the classes for type checking.

[To be updated]

Using IMAP on Mew

2005-12-18T14:10:18Z

If you are using the department Web IMAP, you'll possibly want to leave mails on your mail spool. In this situation, you may want to use IMAP on Mew too. Here, I'll explain how to setup Mew to use IMAP over SSL. For the general information on how to setup Mew, please read another page too.

Stunnel

To use IMAL over SSL, you need to use stunnel (http://www.stunnel.org/). Mew supports both Stunnel-3.x and Stunnel 4.x. Stunnel is already installed on junkfood machine, so you can just add /usr/imports/stunnel/sbin to your $PATH environment variable to use it.

Import certificate

To use SSL properly, you need to install a department certificate. This page explains general information on the department SSL certificate, and what you need to do is to get the cert file and put it at the proper location with the proper name as follows.

Open this page with your browser. (You may see the warning if you haven't set up your browser.)
Save the file cacert.cer linked from this page.
Run the following command:
```
$ openssl x509 -hash -noout -in cacert.cer
```
This outputs a string like "a0208049". The cert file should be renamed to this string + ".0" and be stored in ~/.certs.
```
$ mkdir ~/.certs
$ mv cacert.cer ~/.certs/`openssl x509 -hash -noout -in cacert.cer`.0
```

You can confirm stunnel is working by the following command (where "nnnnn" is an arbitrary unused port number.)

$ stunnel -c -f -d localhost:nnnnn -a ~/.certs -v 1 -D debug -P none -r ringding.cs.umd.edu:993

then on the second console,

$ telnet localhost nnnnn

you will see a verification error on the first console if a cert is not installed properly. Otherwise you will see an IMAP prompt on the second console. If necessary, more information on setting up a CA certificate is found here.

(Optional) Installing Stunnel

Just in case you need to install stunnel by yourself, below is a summary of commands for installing Stunnel-3.26. (I have written this because I didn't find stunnel on junkfood machine first:) After it is completed, you need to add stunnel to your PATH environment variable. In the case above, stunnel would have be installed in $HOME/local/sbin. (You can change the directory name as you wish, of course.)

$ cd (somewhere)
$ wget http://www.stunnel.org/download/stunnel/src/stunnel-3.26.tar.gz
$ gzip -d < stunnel-3.26.tar.gz | tar xf -
$ cd stunnel-3.26
$ ./configure --prefix=$HOME/local
$ make (You will be asked to enter your DN during compiling. Just come up with something and enter it.)
$ make install

IMAP folder setup

The current department IMAP server makes an entire home directory visible to a client. Under this setting, Mew tries to scan the whole files in ~/ when creating the folder index by default, which is time-consuming and useless. One way to avoid this is to make Mew use an "MH directory" setting instead. To allow the IMAP server to use this mode, you need to create a file ~/.mh_profile and write just one line in it:

Path: Mail

(The file .mh_profile is originally a user configuration file for mh. Here I omit all backgrounds for why this is necessary for IMAP.)

Mew configuration - pop3-like usage

To use IMAP, the content of the file ~/.mew.el should be changed. For a certain reason, the setting in this section is not a common configuration for using imap. (I'll describe an alternative setting in the next section. Take a look at it too, and choose the one you prefer.) The configuration in this section makes Mew use imap in a "pop3-like" manner.

(setq mew-imap-prefix-list '("#mh/" "#mhinbox"))
(setq mew-use-cached-passwd t)
;; Uncomment the following line if you don't want to get mail when startup
;(setq mew-auto-get nil)
(setq mew-config-alist
      '(
	("default"
	 ("name"        . "Your Name")
	 ("mail-domain" . "cs.umd.edu")
	 ("proto" . "+")
	 ("mailbox-type" . imap)
	 ("imap-server" . "ringding.cs.umd.edu")
	 ("imap-ssl" . t)
	 ("imap-ssl-port" . "993")
	 ("imap-size" . 0)
	 ("imap-delete" . t))
	("local"
	 ("mailbox-type"     . mbox)
	 ("mbox-command"     . "incm")
	 ("mbox-command-arg" . "-c -d /var/mail/username"))))
(setq mew-nntp-server "news.umd.edu")

Once it's done, restart emacs and Mew. You will have to press "3Z" ("3", then Shift+"z"), then press enter to update folder list cache when you use IMAP for the first time. You'll be asked a password because now Mew is connecting to the IMAP server instead of reading from the local spool. When you start Mew or enter "i" in the summary mode, you will be asked a password because now Mew is retrieving mails from the imap server instead of local spool. Except that, everything is the same. Outgoing mails are sent to local host. Read the later section about the password management. Note that the mails will be deleted on the server when you retrieve mail. If you want the server to keep the retrieved mails, set mew-imap-delete to t.

	 ("imap-delete" . nil)

Mew configuration - general imap

IMAP provides much more powerful remote mail management than POP3. To utilize it (really use IMAP), a differnt setting is necessary. Please keep several things in mind, though. First, you should be aware about "remote folder" versus "local folder". This will change the way you use Mew a little bit. Second, the current imap system on the junkfood system does not support such an advanced functions. Until the system is updated, you can't do much better than the previous setting. Take a look at it too, and choose what you prefer. The ~/.mew.el should look like:

(setq mew-imap-prefix-list '("#mh/" "#mhinbox"))
(setq mew-use-cached-passwd t)
;; Uncomment the following line if you don't want to get mail when startup
;(setq mew-auto-get nil)
(setq mew-config-alist
      '(
	("default"
	 ("name"        . "Your Name")
	 ("mail-domain" . "cs.umd.edu")
	 ("proto" . "%")
	 ("imap-server" . "ringding.cs.umd.edu")
	 ("imap-ssl" . t)
	 ("imap-ssl-port" . "993")
	 ("imap-size" . 0)
	 ("imap-delete" . nil))
	("local"
	 ("proto" . "+")
	 ("mailbox-type"     . mbox)
	 ("mbox-command"     . "incm")
	 ("mbox-command-arg" . "-c -d /var/mail/username"))))
(setq mew-nntp-server "news.umd.edu")

Once it's done, restart emacs and Mew. You'll be asked to press "3Z" ("3", then Shift+"z"), as you did in the first method. After Mew is started, you can retrieve mails from the server by entering "s", then enter. You don't use "i" in the IMAP directory. The retrieved mails are stored in the directory such as ~/Mail/.imap/username\@ringding.cs.umd.edu%imap/. As this name implies, the files in this directory is no longer considered as "real entity". They are just caches of actual mails on the IMAP server.

Copy/move mails to local

Sometimes you may want to copy or move your mail on the spool to your local directory. (Here, "local" means the ~/Mail/inbox directory, for example. Unlike directories under ~/Mail/.imap, a mail in this directory is not a cache.) The quickest way to move mails to local is to enable the local access to the spool and use "i" command. This can be done by the following step. (Caution: after this operation, all mails in the spool are moved to local, so the spool will become empty!)

In the summary mode, enter "C"
In the mini-buffer, enter "local" and press the enter key.
Press "i" to retrieve mails from the spool to the local directory
In the summary mode, enter "C"
In the mini-buffer, enter "default" and press the enter key. Now "i" key is disabled.
To go back to the imap folder, press "g" in the summary mode, and enter "%inbox" then enter. To go to the local inbox, press "g" and enter "+inbox" then enter. (This is part of Mew's folder mechanism. You will understand more if you read the manual about how to refiling messages into folders. here I explained the very minimum stuffs. You can copy or move individual mails too, of course.)

SMTP

A general instruction on authenticated SMTP is found here. In short, what you need to do is a TLS tunnel on either port 25 or 587. (Port 587 is prepared for the access from providers blocking outbound port 25.) Below is the setting you need for the port 25, together with the general imap settings. (Of course, smtp over TLS can be used with POP3-like imap too.) Note that I have added the line "imap-user" in case your junkfood account name is different from the one you use on your pc.

(setq mew-imap-prefix-list '("#mh/" "#mhinbox"))
(setq mew-use-cached-passwd t)
;; Uncomment the following line if you don't want to get mail on startup
(setq mew-auto-get nil)
(setq mew-config-alist
      '(
	("default"
	 ("name"        . "Your Name")
	 ("mail-domain" . "cs.umd.edu")
	 ("proto" . "%")
	 ("smtp-auth-list" . ("PLAIN"))
	 ("smtp-server" . "smtp.cs.umd.edu")
	 ("smtp-port" . "25")
	 ("smtp-ssl"    . t)
	 ("smtp-ssl-port" . "25")
	 ("smtp-user"   . "username")
	 ("imap-server" . "ringding.cs.umd.edu")
	 ("imap-ssl" . t)
	 ("imap-ssl-port" . "993")
	 ("imap-user"   . "username")
	 ("imap-size" . 0)
	 ("imap-delete" . nil))
	; The settings for local access may not be necessary for your pc, but
	; I just leave it here.
	("local"
	 ("proto" . "+")
	 ("smtp-auth-list" . ("CRAM-MD5" "PLAIN" "LOGIN"))
	 ("smtp-server" . "localhost")
	 ("smtp-ssl"    . nil)
	 ("mailbox-type"     . mbox)
	 ("mbox-command"     . "incm")
	 ("mbox-command-arg" . "-c -u -d /var/mail/username"))))
(setq mew-nntp-server "news.umd.edu")

About password

Mew remembers on the memory the password you entered for a short period of time - about 20 minutes. After you didn't use the password during this time period, the password is cancelled and you have to reenter the password. This design is a result of tradeoff between usability and security. There are ways to customize this behavior.

If you don't want Mew to remember the password, set mew-use-cached-passwd to nil:
```
(setq mew-use-cached-passwd nil)
```
If you want Mew to remember the password for longer time period, you can add something like the following to .mew.el. (This example makes Mew remember the password practically long enough.)
```
(setq mew-passwd-timer-unit 999)
(setq mew-passwd-lifetime 999)
```
If you really want to, you can write the password in your .mew.el. I don't recommend it for security reason. Entering password one time when you start Mew is not really frustrating at all. If you wrote password in your .mew.el, however, make sure that file permission is set correctly so that the password is not easily readable from others. (chmod 600 ~/.mew.el*)
```
(setq mew-passwd-alist '(("username@ringding.cs.umd.edu:imap" "password" 0)))
```