2D Graphics and User Interfaces

Why graphics first?

Everything on the screen is graphics.  Everything.

Without understanding how things get drawn on the screen, one can not understand how interfaces are created, and one can not create new kinds of interfaces at all.

The drawing abstraction

Every computer graphics system (i.e., operating system) offers some notion of a canvas to draw onto. 

Usually, the canvases are separated into windows that are distinct from each other, which provide a relative coordinate system and isolation.  This abstraction allows the same binary-level application to render onto different kinds of surfaces such as screens, off-screen buffers (i.e., clipboards), and printers, plotters, and direct neural implants...

Most current systems offer a a resolution-independent (or device-independent) API.  This is crucial.  If all rendering was done in pixels, a 100 pixel rectangle would be about an inch big on most displays, but would be less than 1/10th of an inch big on some printers.

Java offers the Graphics/Graphics2D classes that present this abstraction, and C# offers the System.Drawing.Graphics class.

Coordinates

Device coordinates - Coordinates of the physical device:

• Usually has origin (0, 0) at upper left
• X increases to the right
• Y increases down
• Integral coordinates always has X in  [0, width] and Y in [0, height]
• Coordinates correspond to pixels
• Need to know pixel density (DPI - dots per inch) to create a specific-sized object

Window coordinates - Coordinates within an operating system window

• Similar to device coordinates
• Can have out of bounds coordinates which are clipped by the OS
• If there is a frame around the window ("window dressing"), that is outside the dimensions of the window

Physical coordinates ("device-independent coordinates") - correspond to physical measurements

• Defined in universal measurements - inches, millimeters, points (1/72 of an inch)
• Necessary to make graphics the same size independent of device

Model ("local") coordinates - correspond to the object you are defining

• You may want to define an application-specific coordinate system
• I.e., Origin at the center of screen with one "unit" per inch
• Need to convert between coordinate systems

Transformations

• A "transform" object encapsulates a coordinate system
• A sequence of transforms can efficiently switch coordinate systems

Graphics description models

Stroke model describes images with strokes of specified color and thickness.  There are also several other parameters, such as how the line segments are connected, and whether the line is drawn solid, or with dashes.  In addition, strokes can be anti-aliased.

Line from (4,5) to (9,7) in red with a thickness of 5
Circle centered at (19,8) in blue with a radius of 8 and a thickness of 3
...

 
Miter Join

Round Join

Bevel Join

Antialiasing a line

         
Antialiasing a line

Region model describes images with filled areas such as arcs, text, splines, and other shapes - often other stroke objects.  The area may be filled with a color, or a more complex fill such as a gradient or texture (known as a paint in Java or a Brush in C#.)

Polygon filling (0, 0)-(10, 5)-(5, 10) with yellow
...

Solid fill:    Gradient fill:

Pixel model describes images as a discrete number of pixels. Each pixel is specified by a color from one of several possible color models.

In practice, graphics are described with a combination of stroke, region, and pixel descriptions.

Graphics objects

Graphics object (device context) provides an interface between the application program and the display device. An uniform set of graphics primitives is provided for very different devices: display screen, printer, image file, ...

In C#, there is a single Graphics class, and you access it through another class.

protected override void OnPaint(PaintEventArgs e) {
    Graphics g = e.Graphics;
    g.DrawString("Hello World!", font, blackBrush, 100, 100);
}  

In Java AWT (abstract windowing toolkit) the Graphics object is an instance of the class java.awt.Graphics. It is normally obtained as a parameter to the window update function.

public void paint(Graphics g) {
  g.setColor(Color.red);
  g.fillRect(10, 10, 100, 200);

  Graphics2D g2 = (Graphics2D)g;   // g is actually an instance of a Graphics2D
  g2.draw(...);
}

In both cases, Graphics holds state that prescribes how future Graphics method calls will actually behave.

There is a difference in Java & C# between what state Graphics holds, and what values are passed in as parameters to the render methods.

Example:

C# Graphics does not have Color state, but instead requires Pen and Brush objects passed as parameters

Pen redPen = new Pen(Color.Red);
g.DrawLine(redPen, 0, 5, 10, 15);
g.DrawLine(redPen, 50, 50, 30, 20);

Java Graphics has Color state, and uses a Color for stroke and fill (of Graphics), and uses a Paint for fill (of Graphics2D)

g.setColor(Color.red);
g.drawLine(0, 5, 10, 15);
g.drawLine(50, 50, 30, 20);

What are the tradeoffs between these two approaches?

Which state is stored in each kind of Graphics?

How to do it

/*
 * A Java program that creates a custom component with its own renderer.
 *
 * Ben Bederson, January 29, 2002
 */

import java.awt.*;
import java.awt.event.*;
import java.awt.geom.*;
import javax.swing.*;

public class BasicGraphics extends JFrame {
    static public void main(String[] args) {
        new BasicGraphics();
    }

    public BasicGraphics() {
        setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);

        BCanvas canvas = new BCanvas();
        canvas.setPreferredSize(new Dimension(400, 400));

        getContentPane().add(canvas);
        pack();
        setVisible(true);
    }
}

class BCanvas extends JComponent {
    public void paintComponent(Graphics graphics) {
        Graphics2D g2 = (Graphics2D) graphics;

        g2.setColor(Color.white);
        g2.fillRect(getX(), getY(), getWidth(), getHeight());

        g2.setFont(new Font("Helvetica", Font.PLAIN, 15));
        g2.setColor(Color.black);
        g2.drawString("Hello World!", 100, 100);
    }
}

Java Basic Graphics Rendering - BasicGraphics.java

class StrokeCanvas extends JComponent {
    public void paintComponent(Graphics graphics) {
        Graphics2D g2 = (Graphics2D) graphics;

        g2.setColor(Color.white);
        g2.fillRect(getX(), getY(), getWidth(), getHeight());

	Stroke stroke = new BasicStroke(10, BasicStroke.CAP_ROUND, 
		BasicStroke.JOIN_ROUND);
	g2.setStroke(stroke);
        g2.setColor(Color.red);
        
        Rectangle2D rect = new Rectangle2D.Double(10, 10, 100, 100);
        Line2D line = new Line2D.Double(50, 50, 200, 100);

        g2.draw(rect);
        g2.draw(line);
        
        g2.setRenderingHint(RenderingHints.KEY_ANTIALIASING, 
		RenderingHints.VALUE_ANTIALIAS_ON);
	Stroke stroke2 = new BasicStroke(10, BasicStroke.CAP_BUTT, 
		BasicStroke.JOIN_MITER);
	g2.setStroke(stroke2);
        Line2D line2 = new Line2D.Double(200, 50, 50, 100);
	g2.draw(line2);
    }
}

Java Stroke Rendering - BasicStroker.java

class RegionCanvas extends JComponent {
    public void paintComponent(Graphics graphics) {
        Graphics2D g2 = (Graphics2D) graphics;

        g2.setColor(Color.white);
        g2.fillRect(getX(), getY(), getWidth(), getHeight());

        Paint paint = Color.blue;
        g2.setPaint(paint);
        Rectangle2D rect = new Rectangle2D.Double(10, 10, 100, 100);
        g2.fill(rect);

        Paint paint2 = new GradientPaint(100, 100, Color.red, 200, 200, 
		Color.blue);
        Rectangle2D rect2 = new Rectangle2D.Double(100, 100, 100, 100);
        g2.setPaint(paint2);
        g2.fill(rect2);
    }
}

Java Basic Region Rendering - BasicRegion.java

/*
 * A Java program that creates a custom component with its own renderer,
 * and demonstrates how to load and render images.
 *
 * Ben Bederson, January 29, 2002
 */

import java.awt.*;
import java.awt.event.*;
import java.awt.geom.*;
import javax.swing.*;

public class BasicImage extends JFrame {
    static public void main(String[] args) {
        new BasicImage();
    }

    public BasicImage() {
        setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);

        ImageCanvas canvas =
            new ImageCanvas("dana.jpg");
        canvas.setPreferredSize(new Dimension(400, 400));

        getContentPane().add(canvas);
        pack();
        setVisible(true);
    }
}

class ImageCanvas extends JComponent {
    Image image = null;

    public ImageCanvas(String fileName) {
        loadImage(fileName);
    }

    public void loadImage(String fileName) {
        image = Toolkit.getDefaultToolkit().createImage(fileName);

        MediaTracker tracker = new MediaTracker(this);
        tracker.addImage(image, 0);
        try {
            tracker.waitForID(0);
        } catch (InterruptedException exception) {
            System.out.println("Couldn't load image: " + fileName);
        }
    }

    public void paintComponent(Graphics graphics) {
        Graphics2D g2 = (Graphics2D) graphics;

        g2.setColor(Color.white);
        g2.fillRect(getX(), getY(), getWidth(), getHeight());

        g2.drawImage(image, 50, 50, this);
    }
}

Java Basic Image Rendering - BasicImage.java

/*
 * A C# program that creates a custom component with its own renderer.
 *
 * Ben Bederson, January 30, 2002
 */

using System;
using System.Drawing;
using System.Windows.Forms;

public class BasicGraphics : System.Windows.Forms.Form {
    static public void Main() {
	Application.Run(new BasicGraphics());
    }

    public BasicGraphics() {
	Size = new Size(400, 400);
	StartPosition = FormStartPosition.CenterScreen;
        BasicControl control = new BasicControl();
	control.Location = new Point(0, 0);
	control.Size = new Size(400, 400);

	Controls.Add(control);
    }
}

class BasicControl : Control {
    static Brush blackBrush = new SolidBrush(Color.Black);
    static Font font = new Font("Arial", 15);

    protected override void OnPaint(PaintEventArgs e) {
        Graphics g = e.Graphics;
	BackColor = Color.White;
	g.DrawString("Hello World!", font, blackBrush, 100, 100);
    }
}

C# Basic Graphics Rendering

class StrokeControl : Control {
    static Pen redPen = new Pen(Color.Red);

    protected override void OnPaint(PaintEventArgs e) {
        Graphics g = e.Graphics;
	BackColor = Color.White;

	redPen.LineJoin = System.Drawing.Drawing2D.LineJoin.Round;
	redPen.StartCap = System.Drawing.Drawing2D.LineCap.Round;
	redPen.EndCap = System.Drawing.Drawing2D.LineCap.Round;
	redPen.Width = 10;
	g.DrawRectangle(redPen, 10, 10, 100, 100);
	g.DrawLine(redPen, 50, 50, 200, 100);

	redPen.StartCap = System.Drawing.Drawing2D.LineCap.Flat;
	redPen.EndCap = System.Drawing.Drawing2D.LineCap.Flat;
	g.SmoothingMode = System.Drawing.Drawing2D.SmoothingMode.AntiAlias;
	g.DrawLine(redPen, 200, 50, 50, 100);
    }
}

C# Basic Stroke Rendering

class RegionControl : Control {
    protected override void OnPaint(PaintEventArgs e) {
        Graphics g = e.Graphics;
        BackColor = Color.White;

        g.FillRectangle(Brushes.Blue, 10, 10, 100, 100);

        Brush brush = new LinearGradientBrush(new Point(0, 0), 
		new Point(100, 100), Color.Red, Color.Yellow);
        g.FillRectangle(brush, 150, 10, 100, 100);
    }
}

C# Basic Region Rendering

class ImageControl : Control {
    private Image image;

    public Image Image {
        get { return image; }
        set { 
            image = value;
            Invalidate();
        }
    }

    protected override void OnPaint(PaintEventArgs e) {
        Graphics g = e.Graphics;
        BackColor = Color.White;

        g.DrawImage(image, 0, 0);
    }
}

C# Basic Image Rendering

C# Rendering Tester - PaintingTester.zip

Drawing shapes

There is also support for geometric shapes.

Java Graphics2D has generic Shape class with g2.draw() and g2.fill() methods.  See java.awt.geom. Including GeneralPath which connects points w/ lines or curves.  It can be rendered resolution-independent, or can be flattened with FlatteningPathIterator

C# has fixed shapes rather than a generic Shape class.  See Graphics.DrawEllipse, DrawBezier, DrawCurve, etc.  C# also has a generic path called Drawing2D.GraphicsPath, rendered with Graphics.DrawPath.

Rendering - Damage/Redraw

Override the appropriate paint method of your canvas.

Render when you are asked to.

Request to be rendered with JComponent.repaint()  (Java)  or Control.Invalidate (C#).  Remember that these only request repaints to happen in the future, they do not happen immediately.

Or, just repaint a portion of the canvas - repaint(x, y, w, h)    (Java)  or  Invalidate(Rectangle) C#

Be careful to not to request a repaint within a paint method.  Why?

Text

A special kind of graphics for both performance and human reasons.

Fonts can be bitmaps or curves (or bitmaps generated from curves)

Typically allocate them up front, and then use them on demand.

Java & C# have a Font class

Java

g.setFont(font);
g.setColor(color);
g.drawString(string, x, y);

C#

g.drawString(string, font, brush, point);

Strings get drawn with dimensions (FontMetrics class, g.getFontMetrics() - Java), (Graphics.MeasureString - C#)

How big is a point?

Fonts get measured with Ascent, Descent, Height, Leading, Width, Kerning

Higher quality text can be drawn with antialiasing or more recently, sub-pixel antialiasing (e.g., Microsoft Cleartype)

Color

True color (16, 24, 32 bit) vs. indexed color (8 bit)

Not reflected in the API, rather the API must support the color representation of the hardware.

RGB, HSV, CMYK true-color models

Clipping

Crucial for a window manager

Necessary for application efficiency

Necessary for high-end graphics

Regions - analytical descriptions of shape.  Include algebra for adding/subtracting, manipulating shape.

Java - Area class (made of Shapes)

Area area = new Area(new Rectangle2D.Double(10, 10, 50, 50));
area.intersect(new Area(new Ellipse2D.Double(30, 30, 50, 50));
g.setClip(area);
  // or
g.setPaint(paint);
g.fill(area);

C# - Region class

Region region = new Region(new Rectangle(10, 10, 50, 50));
GraphicsPath path = new GraphicsPath();
path.AddEllipse(30, 30, 50, 50);
g.Clip = region;
  // or
g.FillRegion(brush, region);

Efficiency

Region management (partial redraws)

High-level descriptions for networked displays

Display lists for complex objects that are redrawn many times

Space-time trade-offs.  Ex: pre-allocate thumb image and store it, or render it each time?