[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

10. vgui: Graphical User Interface

Chapter summary: vgui is a user interface library for computer vision applications vgui supports the following general functions:

The vgui design is based on the OpenGL graphics library, and is intended to be platform independent and adaptable to a wide range of GUI toolkits. The central vgui class is the tableau which is a region (or regions) of the screen for carrying out display and event processing. Various tableaux can be assembled and layered to create a complex GUI application. At the same time, each tableau is relatively simple and can often be used independently in a small application such a popup image displayer.

This chapter is concerned with basic vgui programming and does not consider the issues associated with adapting vgui to a new window system and GUI toolkit. The examples are demonstrated using the mfc implementation of vgui.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

10.1 A First Example

A simple example will be useful to illustrate some of the basics of vgui. The appearance of an image displayer is shown in Figure 1.

image-tableau-with-status
Figure 1: A vgui image display application.

The program, basic01_display_image.cxx that produced this display is provided in the examples directory of the vgui library, vgui/examples/. The code is reproduced below.

 
#include <vcl_iostream.h>
#include <vgui/vgui.h>
#include <vgui/vgui_image_tableau.h>
#include <vgui/vgui_viewer2D_tableau.h>
#include <vgui/vgui_shell_tableau.h>
int main(int argc, char **argv)
{
  vgui::init(argc, argv);
  if (argc <= 1)
  {
    vcl_cerr << "Please give an image filename on the command line\n";
    return 0;
  }

  // Load image (given in the first command line param)
  // into an image tableau.
  vgui_image_tableau_new image(argv[1]);

  // Put the image tableau inside a 2D viewer tableau (for zoom, etc).
  vgui_viewer2D_tableau_new viewer(image);

  // Put a shell tableau at the top of our tableau tree.
  vgui_shell_tableau_new shell(viewer);

  // Create a window, add the tableau and show it on screen.
  return vgui::run(shell, image->width(), image->height());
}

It will be useful to go through this example carefully since it brings out some of the important characteristics of programming with tableaux.

In programming with tableaux, it is often necessary to retrieve a particular tableau in a hierarchy such as the shell → viewer → image stack in the example. This access is provided by the method, vgui_tableau_sptr::vertical_cast(vgui_tableau_sptr const& tab). The following code fragment will illustrate its use:

 
vgui_image_tableau_sptr get_image_tab(vgui_tableau_sptr const& tab)
{
  vgui_image_tableau_sptr i_tab;
  if (tab)
    itab.vertical_cast(vgui_find_below_by_type_name(tab,
                       vcl_string("vgui_image_tableau")));
  return i_tab;
}

If the input tableau, tab, is above an image tableau in the hierarchy then this routine will return it, otherwise the returned tableau will be null. One can also keep smart pointers to each tableau as members in an application class, which provides convenient access.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

10.2 Tableaux des Tableaux


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

10.2.1 The Grid

In many computer vision applications, it is useful to be able to display multiple images in the same window. The vgui_grid_tableau is designed for this purpose. The each element of the grid holds a sub-hierarchy of tableaux. A simple example of multiple panes is provided by vgui/examples/basic01a_multiple_panes.cxx, reproduced below. The result of executing this program with two image paths supplied on the command line is shown in Figure 3.

 
#include <vcl_iostream.h>
#include <vcl_algorithm.h>
#include <vgui/vgui.h>
#include <vgui/vgui_image_tableau.h>
#include <vgui/vgui_viewer2D_tableau.h>
#include <vgui/vgui_shell_tableau.h>
#include <vgui/vgui_grid_tableau.h>
int main(int argc, char **argv)
{
  vgui::init(argc, argv);
  if (argc <= 2)
  {
    vcl_cerr << "Please give two image filenames on the command line\n";
    return 0;
  }
  // Load two images(given in the first two command line args)
  // and construct separate image tableaux
  vgui_image_tableau_new image_tab1(argv[1]);
  vgui_image_tableau_new image_tab2(argv[2]);

  //Put the image tableaux into viewers
  vgui_viewer2D_tableau_new viewer1(image_tab1);
  vgui_viewer2D_tableau_new viewer2(image_tab2);

  //Put the viewers into a grid
  vgui_grid_tableau_sptr grid = new vgui_grid_tableau(2,1);
  grid->add_at(viewer1, 0,0);
  grid->add_at(viewer2, 1,0);
  // Put the grid into a shell tableau at the top the hierarchy
  vgui_shell_tableau_new shell(grid);

  // Create a window, add the tableau and show it on screen.
  int width = image_tab1->width() + image_tab2->width();
  int height = vcl_max( image_tab1->height(), image_tab2->height() );
  return vgui::run(shell, width, height);
}

multiple-panes

Figure 3: The vgui_grid_tableau supports the display of multiple panes.

The use of tableaux is very similar to the first example, except that two image tableaux and two viewer2D tableaux are constructed. A 2x1 vgui_vgrid_tableau is constructed and the viewers are inserted in the left and right panes. Note that the order of the indices in the method, vgui_grid_tableau::add_at(unsigned col, unsigned row) is transposed from the order normally used for matrices, i.e. rows then columns. constructed.

In the case of multiple panes, it becomes an issue as to which pane is considered active. That is, suppose we wanted to replace the image in a pane selected by the user. How does the user indicate what pane is to be updated? A simple approach would be to have the user input the column and row of the grid cell to be updated using a menu (we will discuss menus in a later section). There are several additional grid methods that help define the grid cell that is to be operated on.

An application can then use these selections to operate on the desired pane. For example, if a user wants to load and image from a file into a particular pane, they would click on the desired pane and then push the load-image menu. The menu callback routine would use the last_selected_position method to identify the appropriate vgui_image_tableau.

The grid tableau responds to other events as follows:

 
Modifier   Key                        Result
            =        Add a column to the grid
            -        Remove a column from the grid
 CNTL       =        Add a row to the grid
 CNTL       -        Remove a row from the grid
          PAGEUP     Page the grid tablaux forward in the active cell
          PAGEDOWN   Page the grid tablaux backward in the active cell

[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

10.2.2 The Deck

deck-tableau

Figure 4: The vgui_deck_tableau supports the display of a stack of child tableaux. A typical application is to be able to page through a sequence of images. Only the tableau root on the top of the stack responds to events, such as pan and zoom.

Another useful capability is to stack displays in a pane. Then the application can “page” through the displays. Only the tableau hierarchy on the “top” of the deck responds to events, such as zooming and panning controls. The creation of a deck is illustrated by the example, vgui/examples/basic01b_deck.cxx. The concept of a deck is shown in Figure 4. The example code is:

 
#include <vcl_iostream.h>
#include <vnl/vnl_math.h>
#include <vgui/vgui.h>
#include <vgui/vgui_image_tableau.h>
#include <vgui/vgui_viewer2D_tableau.h>
#include <vgui/vgui_shell_tableau.h>
#include <vgui/vgui_deck_tableau.h>

int main(int argc, char **argv)
{
  vgui::init(argc, argv);
  if (argc <= 2)
  {
    vcl_cerr << "Please give two image filenames on the command line\n";
    return 0;
  }
  // Load two images(given in the first two command line args)
  // and construct separate image tableaux
  vgui_image_tableau_new image_tab1(argv[1]);
  vgui_image_tableau_new image_tab2(argv[2]);

  //Put the image tableaux into a deck
  vgui_deck_tableau_sptr deck = vgui_deck_tableau_new();
  deck->add(image_tab1);
  deck->add(image_tab2);

  vgui_viewer2D_tableau_new viewer(deck);

  // Put the deck into a shell tableau at the top the hierarchy
  vgui_shell_tableau_new shell(viewer);

  // Create a window, add the tableau and show it on screen.
  int width = vnl_math_max(image_tab1->width(), image_tab2->width());
  int height = vnl_math_max(image_tab1->height(), image_tab2->height());

  //Add 50 to account for window borders
  return vgui::run(shell, width+50, height+50);
}

The deck tableau responds to vgui_PAGE_UP and vgui_PAGE_DOWN events, which advance or backup the deck sequence.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

10.3 Displaying Geometry


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

10.3.1 Displaying 2-d Features

display-2d-annotated

Figure 5: Geometry overlaid on the image is shown on the right. The tableau graph for this example is shown on the left.

vgui supports the subject-view programming pattern. That is, there is a clear separation between a class object, such as a line segment, and its view which is the manner in which it is rendered on the screen. Indeed, one can have many different views for a given object. The same line can be displayed with different line widths or colors, or even in an entirely different form such as a point in an image of Hough space, (rho, theta). Note, this subject-view approach is perversely called document-view in MFC.

The following example illustrates the ability of vgui to display geometric figures, and its result is shown in Figure 5.

 
#include <vcl_iostream.h>
#include <vgui/vgui.h>
#include <vgui/vgui_image_tableau.h>
#include <vgui/vgui_easy2D_tableau.h>
#include <vgui/vgui_viewer2D_tableau.h>
#include <vgui/vgui_shell_tableau.h>

int main(int argc, char **argv)
{
  vgui::init(argc, argv);
  if (argc <= 1)
  {
    vcl_cerr << "Please give an image filename on the command line\n";
    return 0;
  }

  // Load an image into an image.tableau
  vgui_image_tableau_new image(argv[1]);

  // Put the image.tableau into a easy2D tableau
  vgui_easy2D_tableau_new easy2D(image);

  // Add a point, line, and infinite line
  easy2D->set_foreground(0,1,0);
  easy2D->set_point_radius(5);
  easy2D->add_point(10, 20);

  easy2D->set_foreground(0,0,1);
  easy2D->set_line_width(2);
  easy2D->add_line(100,100,200,400);

  easy2D->set_foreground(0,1,0);
  easy2D->set_line_width(2);
  easy2D->add_infinite_line(1,1,-100);

  // Put the easy2D tableau into a viewer2D tableau:
  vgui_viewer2D_tableau_new viewer(easy2D);
  vgui_shell_tableau_new shell(viewer);

  // Create a window, add the tableau and show it on screen:
  return vgui::run(shell, image->width(), image->height());
}

The first new code we encounter in the example is:

 
  vgui_easy2D_tableau_new easy2D(image);

or equivalently,

 
  vgui_easy2D_tableau_sptr easy2D = vgui_easy2D_tableau_new(image);

The vgui_easy2D_tableau is responsible for rendering 2-d geometric shapes on top of its child, a vgui_image_tableau. The commands for inserting various geometric elements are of the form add_xxx(...). vgui_easy2D_tableau assumes an elemental form of geometric specification, where the points and lines are directly specified by their parameters. The definitions for each add method used in the example are:

 
vgui_soview2D_point* add_point(float x, float y)
vgui_soview2D_lineseg* add_line(float x0, float y0, float x1, float y1)
vgui_soview2D_infinite_line* add_infinite_line(float a, float b, float c)

The point is defined by its location. The line segment is specified by the endpoints. The infinite line is specified by its line coefficients:       ax + by +c = 0.

The appearance (or style) of the display is controlled by the following style specifiers:

 
void set_foreground(float r, float g, float b)
void set_line_width(float w)
void set_point_radius(float r)

The term foreground refers to the color of the displayed geometry. The style of each element added to vgui_easy2D after the set_foreground, set_line_width, and set_point_radius commands is assigned according to their specification until a new style command is issued.

soview-hierarchy

Figure 6: The hierarchy for 2-d soviews. vgui also has some support for 3-d rendering.

A particular view of a geometric entity is specified by the class, vgui_soview which has the hierarchy shown in Figure 6. The constructor for a vgui_soview extracts the necessary information from the object to specify OpenGL rendering commands. The commands then add to the OpenGL display list to be rendered. While the current set is adequate for a wide range of computer vision programming, more advanced users will want to create their own vgui_soview subclass to provide convenient display interfaces for their objects, or to achieve special viewing capabilities.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

10.3.2 Interactive Drawing of 2-d Features

It is often necessary to create geometric features such as a box to define a region of interest for image processing or a line for sampling pixels to provide an intensity plot. The following example shows how to create an interactive tool for drawing lines and circles. Interactive drawing consists of a tight loop of mouse position tracking and rendering so that the feature point tracks the mouse movements. This loop is called rubber-banding, since the feature seems to stretch and pull as the mouse moves. The rubberband loop is usually terminated by an event, such as a left mouse click. An example of a rubberband application is provided in vgui/examples/basic10a_rubberband.cxx. For this example, a left mouse click starts rubber-banding the feature and another left click terminates the rubber-banding and inserts the feature into the vgui_easy2D_tableau.

 
#include <vcl_iostream.h>
#include <vgui/vgui.h>
#include <vgui/vgui_menu.h>
#include <vgui/vgui_image_tableau.h>
#include <vgui/vgui_easy2D_tableau.h>
#include <vgui/vgui_rubberband_tableau.h>
#include <vgui/vgui_viewer2D_tableau.h>
#include <vgui/vgui_shell_tableau.h>

//global pointer to the rubberband tableau
static vgui_rubberband_tableau_sptr rubber = 0;

//the menu callback functions
static void create_line()
{
  rubber->rubberband_line();
}
static void create_circle()
{
  rubber->rubberband_circle();
}

// Create the edit menu
vgui_menu create_menus()
{
  vgui_menu edit;
  edit.add("CreateLine",create_line,(vgui_key)'l',vgui_CTRL);
  edit.add("CreateCircle",create_circle,(vgui_key)'k',vgui_CTRL);
  vgui_menu bar;
  bar.add("Edit",edit);
  return bar;
}

int main(int argc, char ** argv)
{
  vgui::init(argc,argv);
  if (argc <= 1)
  {
    vcl_cerr << "Please give an image filename on the command line\n";
    return 0;
  }
  // Make the tableau hierarchy.
  vgui_image_tableau_new image(argv[1]);
  vgui_easy2D_tableau_new easy(image);
  vgui_rubberband_easy2D_client* r_client =
    new vgui_rubberband_easy2D_client(easy);
  rubber = vgui_rubberband_tableau_new(r_client);
  vgui_composite_tableau_new comp(easy, rubber);
  vgui_viewer2D_tableau_new viewer(comp);
  vgui_shell_tableau_new shell(viewer);

  // Create and run the window
  return vgui::run(shell, 512, 512, create_menus());
}

This example introduces several new coding aspects to discuss.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

10.3.3 Rendering 3-d Features

The 3-d display capabilities in vgui are not well-developed. Currently one can only display 3-d points and lines. The 3-d viewer does provide a “trackball” mode of interaction in viewing the 3-d geometry. An example of the vgui 3-d display is shown in Figure 7.

threeD-example
Figure 7: vgui's 3-d display.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

10.4 Menus


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

10.4.1 Basic Menus

vgui supports basic menu capabilities such as menu bars and popup menus as illustrated in the following example, vgui/examples/basic05_menubar.cxx:

 
#include <vcl_iostream.h>
#include <vgui/vgui.h>
#include <vgui/vgui_menu.h>
#include <vgui/vgui_image_tableau.h>
#include <vgui/vgui_viewer2D_tableau.h>
#include <vgui/vgui_shell_tableau.h>

// Set up a dummy callback function for the menu to call (for
// simplicity all menu items will call this function):
static void dummy()
{
  vcl_cerr << "Dummy function called\n";
}

// Create a vgui menu:
vgui_menu create_menus()
{
  vgui_menu file;
  file.add("Open",dummy,(vgui_key)'O',vgui_CTRL);
  file.add("Quit",dummy,(vgui_key)'R',vgui_SHIFT);

  vgui_menu image;
  image.add("Center image",dummy);
  image.add("Show histogram",dummy);

  vgui_menu bar;
  bar.add("File",file);
  bar.add("Image",image);

  return bar;
}

int main(int argc, char ** argv)
{
  vgui::init(argc,argv);
  if (argc <= 1)
 {
    vcl_cerr << "Please give an image filename on the command line\n";
    return 0;
 }

  // Make our tableau hierarchy.
  vgui_image_tableau_new image(argv[1]);
  vgui_viewer2D_tableau_new viewer(image);
  vgui_shell_tableau_new shell(viewer);

  // Create a window, but this time we also pass in a vgui_menu.
  return vgui::run(shell, 512, 512, create_menus());
}

The appearance of this program is shown in Figure 8.

menu-example
Figure 8: A vgui example involving menus. The menu bar on the top has sub-menus as indicated in the figure. In the example, all the menu choices call the same dummy function.

By now, most of this code should follow a familiar pattern. The new element is the function vgui_menu create_menus(). The menu structure is assembled hierarchically where the top-level menus have sub-menus which can have, sub-menus etc. The basic menu construction pattern is illustrated by the line:

 
  file.add("Open",dummy,(vgui_key)'O',vgui_CTRL);

The first argument "Open" is a string representing the label of the menu item in the menu. The second argument is the name of the function to be called when the menu is selected. The last two arguments define a key-press configuration that will select the menu item without clicking on it with the mouse. In this case, the function dummy() is called by pressing the key combination, CTRL + 'o'.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

10.4.2 Pop-up Menus

The appearance of a vgui pop-up menu is shown in Figure 9. A pop-up menu is launched by pressing the right mouse button over the active application window. Typically the role of the pop-up menu is to present operations that are relevant to the context present when the right button is pressed. For example, if we are displaying an image, there would be image display or image processing operations presented in the menu. Another mode might dominate when the tableau contains only geometric features. In that case, the menu items might present geometric operations such as translation or rotation.

popupmenu-example
Figure 9: A vgui pop-up menu example. The menu shown was launched by pressing the right mouse key. A pop-up menu can have sub-menus, as shown in the figure.

This display was created by the example vgui/examples/basic06_popup.cxx:

 
#include <vcl_iostream.h>
#include <vgui/vgui.h>
#include <vgui/vgui_window.h>
#include <vgui/vgui_adaptor.h>
#include <vgui/vgui_menu.h>
#include <vgui/vgui_image_tableau.h>
#include <vgui/vgui_viewer2D_tableau.h>
#include <vgui/vgui_shell_tableau.h>
// Set up a dummy callback function for the menu to call (for
// simplicity all menu items will call this function):
static void dummy()
{
  vcl_cerr << "Dummy function called\n";
}

// Create a vgui_menu:
vgui_menu create_menus()
{
  vgui_menu file;
  file.add("Open", dummy);
  file.add("Quit", dummy);

  vgui_menu image;
  image.add("Center image", dummy);
  image.add("Show histogram", dummy);

  vgui_menu bar;
  bar.add("File",file);
  bar.add("Image",image);

  return bar;
}
 
int main(int argc, char ** argv)
{
  vgui::init(argc,argv);
  if (argc <= 1)
  {
    vcl_cerr << "Please give an image filename on the command line\n";
    return 0;
  }
  // Load an image into an image tableau:
  vgui_image_tableau_new image(argv[1]);
  vgui_viewer2D_tableau_new viewer(image);
  vgui_shell_tableau_new shell(viewer);

  // Create a window and add the tableau:
  vgui_window *win = vgui::produce_window(512, 512);
  win->get_adaptor()->set_tableau(shell);

  // Add our menu items to the base pop-up
  // (this menu appears when the user clicks
  //  the right mouse button on the tableau)
  win->get_adaptor()->include_in_popup(create_menus());
  win->get_adaptor()->bind_popups();
  win->show();
  return vgui::run();
}

This example contains a few new elements that should be discussed.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

10.5 Dialog Menus

It is often necessary to obtain values for parameters, such as edge detection thresholds or scales before applying the operation to an image. Also it is frequently necessary to obtain a file path string in order to read data such as image. These functions are satisfied by the dialog menu. An example of a dialog is shown in Figure 10, which corresponds to vgui/examples/basic07_dialog.cxx.

dialog-example
Figure 10: The dialog menu appears when a menu item is pressed that calls a function containing a dialog. The dialog displays names and values as well as boolean check boxes. The state of the boxes and values can be changed and when the dialog is dispatched, the altered values are bound to the variables in the dialog specification. In this simple example, the latest values are output to vcl_cerr.

The following example illustrates these points.

 
#include <vcl_iostream.h>
#include <vbl/vbl_bool_ostream.h>
#include <vgui/vgui.h>
#include <vgui/vgui_menu.h>
#include <vgui/vgui_dialog.h>
#include <vgui/vgui_image_tableau.h>
#include <vgui/vgui_viewer2D_tableau.h>
#include <vgui/vgui_shell_tableau.h>
// Make a vgui_dialog:
static void test_dialog()
{
  static int int_value = 2;
  static long long_value = 3;
  static float float_value = 3.1f;
  static double double_value = 4.2;
  static vcl_string string_value = "dialog test";
  static bool bool_value = true;
  static vcl_string file_value = "/tmp/myfile.txt";
  static vcl_string regexp = "*.txt";
  static vcl_string color_value = "blue";

  static int choice_value = 1;
  vcl_vector<vcl_string> labels;
  labels.push_back(vcl_string("fltk"));
  labels.push_back(vcl_string("motif"));
  labels.push_back(vcl_string("gtk"));
  labels.push_back(vcl_string("glut"));
  labels.push_back(vcl_string("glX"));

  vgui_dialog mydialog("My dialog");
  mydialog.field("int value", int_value);
  mydialog.field("long value", long_value);
  mydialog.field("float value", float_value);
  mydialog.field("double value", double_value);
  mydialog.field("string value", string_value);
  mydialog.checkbox("bool value", bool_value);
  mydialog.choice("choice value", labels, choice_value);
  mydialog.inline_file("file browser", regexp, file_value);
  mydialog.inline_color("color value", color_value);

  if (mydialog.ask())
  {
    vcl_cerr << "int_value : " << int_value << vcl_endl
             << "long_value : " << long_value << vcl_endl
             << "float_value : " << float_value << vcl_endl
             << "double_value : " << double_value << vcl_endl
             << "string_value : " << string_value << vcl_endl
             << "bool_value : "
             << vbl_bool_ostream::true_false(bool_value) << vcl_endl
             << "choice_value : " << choice_value << ' '
             << labels[choice_value] << vcl_endl
             << "file_value: " << file_value << vcl_endl
             << "color_value: " << color_value << vcl_endl;
  }
}
 
static void test_dialog2()
{
  vgui_dialog mydialog("My dialog2");
  vgui_image_tableau_new image("c:/house11_small.jpg");
  vgui_viewer2D_tableau_new viewer(image);
  mydialog.inline_tableau(viewer, 512, 512);

  mydialog.message("A picture");

  vcl_string button_txt("close");
  mydialog.set_ok_button(button_txt.c_str());
  mydialog.set_cancel_button(0);
  mydialog.ask();
}

// Create a vgui.menu with an item which shows the dialog box:
vgui_menu create_menus()
{
  vgui_menu test;
  test.add("Dialog", test_dialog);
  test.add("Dialog2", test_dialog2);

  vgui_menu bar;
  bar.add("Test",test);

  return bar;
}

A dialog pops up when the associated menu item is selected. The dialog interface is reasonably self-explanatory, but it will be useful to discuss some of the main elements.

choice-color

Figure 11: a) The choice option enables a selection from a set of alternative values. b) The color option enables a selection from a pallet of colors.

dialog2-example
Figure 12: Dialogs can also contain embedded images or geometric figures.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

10.6 Event Processing

As applications become more sophisticated, the programmer will need to be able to process events in a manner appropriate to customized interactive tasks. The vgui design has a simple interface for handling events as is illustrated by vgui/examples/basic09_mouse_position. The code for this example:

 
#include <vcl_iostream.h>
#include <vgui/vgui.h>
#include <vgui/vgui_image_tableau.h>
#include <vgui/vgui_viewer2D_tableau.h>
#include <vgui/vgui_shell_tableau.h>
//------------------------------------------------------------
// A tableau that displays the mouse position
// when left mouse button is pressed.
struct example_tableau : public vgui_image_tableau
{
  example_tableau(char const* f) : vgui_image_tableau(f){ }

  ~example_tableau() { }

  bool handle(vgui_event const& e)
  {
    if (e.type == vgui_BUTTON_DOWN &&
        e.button == vgui_LEFT && e.modifier == 0)
    {
      vcl_cout << "selecting at " << e.wx << ' ' << e.wy << vcl_endl;
      return true; // event has been used
    }

    //  We are not interested in other events,
    //  so pass event to base class:
    return vgui_image_tableau::handle(e);
  }
};
//-------------------------------------------------------------
// Make a smart-pointer constructor for our tableau.
struct example_tableau_new : public vgui_image_tableau_sptr
{
  example_tableau_new(char const* f) : vgui_image_tableau_sptr(
    new example_tableau(f)) { }
};
 
//-------------------------------------------------------------
//The first command line argument is expected
// to be an image filename.
int main(int argc,char **argv)
{
  vgui::init(argc, argv);
  if (argc <= 1)
  {
    vcl_cerr << "Please give an image filename\n";
    return 0;
  }

  // Load an image into my tableau
  // (derived from vgui_image_tableau)
  vgui_tableau_sptr my_tab = example_tableau_new(argv[1]);

  vgui_viewer2D_tableau_new viewer(my_tab);
  vgui_shell_tableau_new shell(viewer);

  // Start event loop, using easy method.
  return vgui::run(shell, 512, 512);
}

This example contains a number of new concepts that are important to building custom applications. The example illustrates how to go about creating a new tableau, which is a subclass of an existing tableau.

It is often required to include the definition of a tableau smart pointer in other class implementations. When a new tableau is created it is convenient to define the smart pointer in a ‘xxx_tableau_sptr.h’ file as follows:

 
#include <vgui/vgui_tableau_sptr.h>

class xxx_tableau;
typedef vgui_tableau_sptr_t<xxx_tableau> xxx_tableau_sptr;

[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

10.6.1 vgui Events

The general types of events handled by vgui are:

The following is a partial list of the event enum symbols defined in vgui_event.h. Not all of the events enumerated in vgui_event are described here. Some of them seem to be vestigial and not exploited in code. A few comments will be added where the event function is not obvious.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

10.6.2 vgui Buttons, Keys and Modifiers

The following table defines the vgui enum symbols for buttons, keys and modifiers:

===== Buttons =====

 

vgui_LEFT

left mouse button

vgui_MIDDLE

middle mouse button

vgui_RIGHT

right mouse button

===== Keys =====

 

vgui_ESC

ascii 27

vgui_TAB

\t

vgui_RETURN

\r

vgui_NEWLINE

\n

vgui_F1

0x100 + 1

vgui_F2

0x100 + 2

vgui_F3

0x100 + 3

vgui_F4

0x100 + 4

vgui_F5

0x100 + 5

vgui_F6

0x100 + 6

vgui_F7

0x100 + 7

vgui_F9

0x100 + 8

vgui_F10

0x100 + 9

vgui_F11

0x100 + 10

vgui_F12

0x100 + 11

vgui_CURSOR_LEFT

0x100 + 12

vgui_CURSOR_UP

0x100 + 13

vgui_CURSOR_RIGHT

0x100 + 14

vgui_CURSOR_DOWN

0x100 + 15

vgui_PAGE_UP

0x100 + 16

vgui_PAGE_DOWN

0x100 + 17

vgui_HOME

0x100 + 18

vgui_END

0x100 + 19

vgui_DELETE

0x100 + 20

vgui_INSERT

0x100 + 21

===== Modifiers =====

 

vgui_NULL

0x0

vgui_CTRL

0x1

vgui_SHIFT

0x2

vgui_META

0x4

vgui_ALT

0x8

When a key press event is dispatched it carries information that specifies the key and modifier structure. This code fragment illustrates the use of modified keys:

 
bool handle(vgui_event const& e)
{
  vgui_key k = e.key;
  vgui_modifier m = e.modifier;
  if (m & vgui_CTRL)
    if (k == 's')
     {
      // Do something appropriate for CTRL + 's'
      ...
     }
  return true;
}

The bit corresponding to the modifier is tested to see if further action switched by the actual key is warranted. Note that the key in a vgui_event is always lower case. This eliminates the ambiguity that might arise in the use of the SHIFT key and upper case vs lower case characters. The following table will illustrate the effect of various modifier combinations.

key press

modifier

key

ascii character

=======

=======

=======

=======

a

vgui_NULL

a

a

CTRL+a

vgui_CTRL

a

^a

SHIFT+a

vgui_SHIFT

a

A

/

vgui_NULL

/

/

?

vgui_SHIFT

/

?

If one wants to work directly with the actual ascii character pressed, then use e.ascii_char.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

10.6.3 Event Condition

A convenient class, vgui_event_condition is defined to represent the occurrence of a particular event configuration. Its use is best illustrated by an example:

 
bool my_tableau::handle(vgui_event const& e)
{
   vgui_event_condition g0(vgui_LEFT, vgui_CTRL, false);
   if (g0(e))
      vcl_cout << "saw a left mouse button release with CTRL pressed event\n";

   // pass the event back to the parent tableau
   return vgui_my_parent_tableau::handle(e);
}

In this case a test for the indicated event condition is constructed and can be used to filter events passing into a tableau's handle method. The event condition class provides a compact and tidy way of expressing complex logic on modifiers, keys and buttons.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

10.6.4 Mouse Position

An event passes back the position of the mouse when the event occurred. As was illustrated in the basic09_mouse_position example. However this position is in the coordinate system of the display window. Most computer vision applications require positions referenced to the coordinate system of the image being displayed, and expressed in pixels. The class vgui_projection_inspector provides methods for transforming between the window and image coordinate systems.

The transformation is illustrated by this code fragment:

 
// Get X,Y mouse position to display on status bar
// in image coordinates
bool my_image_tableau::handle(vgui_event const& e)
{
  if (e.type == vgui_MOTION && button_down == false)
   {
    float pointx, pointy;
    vgui_projection_inspector p_insp;
    p_insp.window_to_image_coordinates(e.wx, e.wy, pointx, pointy);
    int intx = (int)vcl_floor(pointx), inty = (int)vcl_floor(pointy);
    vgui::out << '(' << intx << ' ' << inty << ")\n";
   }
  return vgui_image_tableau::handle(e);
}

[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

10.7 Building an Application

The essential elements to create a GUI application have been presented. In this section, a typical design for a main program and associated GUI management classes will be described.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

10.7.1 The Manager

There is a tendency to pile a lot of menu callbacks and menu constructors into the main program. It is easy and convenient, but this approach doesn't stand up to evolution of the program over time. The main program quickly becomes hopelessly cluttered with a tangle of processing code, menu callbacks and tableau specifications.

A much better approach is to separate the methods used to process callbacks and event handling into a singleton class called the manager. The manager can be a sub-class of the top-level tableau and thus provide custom processing of events by defining its own ::handle method. The methods on the manager provide the implementation for the menu callbacks.

It is also better to separate the menu construction class from the main program, since menus also tend to grow in number and complexity as the application evolves.

The following example will illustrate these design principles. The manager class looks like:

 
#include <vil/vil_image.h>
#include <vgui/vgui_image_tableau_sptr.h>
#include <vgui/vgui_wrapper_tableau.h>

class basic_manager : public vgui_wrapper_tableau
{
 public:
  ~basic_manager();
  static basic_manager *instance();
  void quit();
  void load_image();
  void init();
  virtual bool handle(vgui_event const&);

 private:
  basic_manager();
  vil_image img_;
  vgui_image_tableau_sptr itab_;
  static basic_manager *instance_;
};

Some elements of the class design require explanation:

The implementation of the basic_manager class is as follows:

 
#include <vcl_cstdlib.h> // for vcl_exit()
#include <vcl_iostream.h>
#include <vil/vil_load.h>
#include <vgui/vgui.h>
#include <vgui/vgui_dialog.h>
#include <vgui/vgui_viewer2D_tableau.h>
#include <vgui/vgui_shell_tableau.h>
#include <vgui/vgui_image_tableau.h>
#include "basic_manager.h"

//static basic_manager instance
basic_manager* basic_manager::instance_ = 0;

//insure only one instance is created
basic_manager *basic_manager::instance()
{
  if (!instance_)
  {
    instance_ = new basic_manager();
    instance_->init();
   }
  return basic_manager::instance_;
}

// constructor/destructor
basic_manager::basic_manager():vgui_wrapper_tableau(){}

basic_manager::~basic_manager(){}

void basic_manager::init()
{
  itab_ = vgui_image_tableau_new();//keep the image tableau handy
  vgui_viewer2D_tableau_sptr viewer = vgui_viewer2D_tableau_new(itab_);
  vgui_shell_tableau_sptr shell = vgui_shell_tableau_new(viewer);
  this->add_child(shell);
}

Most of this code should be clear. One subtle point is the method void basic_manager::init(). When the instance of basic_manager is created, the parent class, vgui_wrapper_tableau, is constructed using its default constructor. After it comes into existence, the rest of the tableau hierarchy can be inserted as a child of basic_manager. With this approach, there is no assumption required about the order of constructors.

 
//the event handler
bool basic_manager::handle(vgui_event const& e)
{
  if (e.key == 'b')
    vgui::out << "I saw a 'b'\n";
  //pass the event to the shell
  return this->child.handle(e);
}

void basic_manager::quit()
{
  vcl_exit(1);
}

void basic_manager::load_image()
{
  vgui_dialog load_image_dlg("Load image file");
  static vcl_string image_filename = "";
  static vcl_string ext = "*.*";
  load_image_dlg.file("Image Filename:", ext, image_filename);
  if (!load_image_dlg.ask())
    return;
  img_ = vil_load(image_filename.c_str());
  itab_->set_image(img_);
}

The manager has a simple basic handle method that looks for the letter ‘b’. All events are then passed to the child (shell) tableau for further processing. This routine could return true on the detection of the ‘b’ event if it were desired not to have any of the child tableaux react.

The methods to support menu callbacks are implemented in the manager. For example, basic_manager::load_image() illustrates the use of a dialog which pops up when the “Load Image” menu is selected.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

10.7.2 The Menus

A menu class is defined to package up the static callback functions required in the vgui_menu assembly. The basic_menu class is:

 
//basic_menus.h
class basic_menus
{
 public:
  static void quit_callback();
  static void load_image_callback();
  static vgui_menu get_menu();
 private:
  basic_menus(){};
};
//basic_menus.cxx
#include <vgui/vgui.h>
#include <vgui/vgui_key.h>
#include <vgui/vgui_modifier.h>
#include <vgui/vgui_menu.h>
#include "basic_manager.h"
#include "basic_menus.h"

//Static menu callback functions

void basic_menus::quit_callback()
{
  basic_manager::instance()->quit();
}

void basic_menus::load_image_callback()
{
  basic_manager::instance()->load_image();
}

//basic_menus definitions
vgui_menu basic_menus::get_menu()
{
  vgui_menu menubar;
  vgui_menu menufile;

  //file menu entries
  menufile.add( "Quit", quit_callback,(vgui_key)'q', vgui_CTRL);
  menufile.add( "Load Image", load_image_callback, (vgui_key)'l', vgui_CTRL);

  //main menu bar
  menubar.add( "File", menufile);
  return menubar;
}

Note that the menu callback functions are paired with methods on the manager. These menus can be exported to other GUI libraries so that the same menu functionality can be re-used. However, keep in mind that under Windows special measures must be taken to export static items.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

10.7.3 The Main Program

The main program for the basic_manager application is:

 
#include <vgui/vgui.h>
#include <vgui/vgui_adaptor.h>
#include <vgui/vgui_window.h>
#include <vgui/vgui_command.h>
#include <vgui/vgui_shell_tableau.h>
#include <vgui/internals/vgui_accelerate.h>
#include "basic_menus.h"
#include "basic_manager.h"

int main(int argc, char** argv)
{
  vgui::init(argc, argv);
  vgui_menu menubar = basic_menus::get_menu();
  unsigned w = 512, h = 512;
  vcl_string title = "REALLY BASIC";
  vgui_window* win = vgui::produce_window(w, h, menubar, title);
  basic_manager* bas = basic_manager::instance();
  win->get_adaptor()->set_tableau(bas);
  win->set_statusbar(true);
  win->enable_vscrollbar(true);
  win->enable_hscrollbar(true);
  win->show();
  return vgui::run();
}

Note that the basic manager instance is attached to the adaptor in order to receive events, by the expression, win->get_adaptor()->set_tableau(bas). Note that the main program is now very simple and will stay uncluttered as the application grows.

Several new features have been included in the construction of this window:


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

10.8 Summary of vgui Tableaux

The following is a summary of the tableau defined in vgui.

active_tableau

This tableau (or rather a tableau derived from it) can appear visible or invisible, and active or inactive by calling toggle_active and toggle_visible.

blackbox_tableau

A tableau for event record and playback.

blender_tableau

To use this tableau make a vgui_image_tableau containing one of the images to blend and a vgui_blender_tableau containing the other. Put them both in a vgui_composite_tableau. Set alpha to be less than one to see the blended image.

clear_tableau

A tableau that performs OpenGL clearing upon receipt of a vgui_DRAW event. It has no child tableau.

composite_tableau

The vgui_composite_tableau class can have any number of children, indexed from 0 upwards. The draw action of vgui_composite_tableau is to draw each of its children, in order, into the current context. Events reaching the vgui_composite_tableau are passed on to each child in turn, till it is handled, so that child 0, the first added, is the "top" tableau.

deck_tableau

For holding an ordered collection of child tableaux, only one of which is passed all events that the vgui_deck_tableau receives. The effect is a flick-book of tableaux where the currently active tableau can be changed using PageUp and PageDown.

displaylist2D_tableau

Display of two-dimensional geometric objects - a builder tableau usually sub-classed.

displaylist3D_tableau

Display of three-dimensional geometric objects - a builder tableau usually sub-classed.

drag_tableau

A drag event occurs when the user moves the mouse with one of the mouse buttons pressed down. In vgui there is no vgui_DRAG event (there is only vgui_MOTION for when the mouse is moving). So if you want to capture drag events you may find this tableau handy.

easy2D_tableau

Easy interface for displaying two-dimensional geometric objects (see vgui_soview2D) such as lines, points, circles, etc. can be added using add, or add_point, add_line, add_circle, etc.

easy3D_tableau

Easy interface for displaying three-dimensional objects (see vgui_soview3D) can be added using add, or add_point, add_line, etc.

enhance_tableau

Magnify/display another tableau in a region around the mouse pointer. Useful for a roaming image-processing sub-window.

function_tableau

Allows a user to insert custom functions that are called when events such as draw, mouse up, motion .. etc occur.

grid_tableau

A tableau that renders its child tableaux as a rectangular grid.

image_tableau

A tableau that renders the given image using an image_renderer.

listmanager2D_tableau

A tableau that manages a set of vgui_displaylist2D_tableau children.

loader_tableau

A tableau which (optionally) loads given values for the projection and modelview matrices before passing control to its child. This is typically used to initialize GL before rendering a scene.

poly_tableau

A tableau which renders its children in sub-rectangles of its viewport. The grid_tableau is a sub-class of poly_tableau.

quit_tableau

A tableau which quits the application on receiving ‘q’ or ESC.

roi_tableau

A tableau which makes an ROI of an image act like a whole image.

rubberband_tableau

A tableau for interactive drawing of lines, circles, boxes, etc.

satellite_tableau

Turns a non-tableau into a multi-tableau, or puts one tableau into two parts of the hierarchy simultaneously. Example: We are displaying two images, each in its own zoomer and we'd like to have a tableau which takes mouse events from one image and draws a line on the other; introduces a "crossover" in the tree which is difficult to handle without vgui_satellite_tableau.

shell_tableau

A shell tableau is a handy collection of things one often wants at the very top of one's tableau hierarchy. It is essentially an acetate with N utility tableaux at the bottom.

text_tableau

A tableau for rendering text. Each piece of text is associated with an integer handle through which it can be retrieved, moved about, changed or removed. This tableau will not display any text unless you have compiled with GLUT.

tview_launcher_tableau

A tableau that pops up tableau tree (tview) on ‘G’.

tview_tableau

Displays a tableau tree.

viewer2D_tableau

A tableau for zooming and panning 2-d renderings.

viewer3D_tableau

A tableau for manipulating 3-d rendered scenes (not completed).

wrapper_tableau

A base class tableau which insures only a single child. Useful as a base class for managers.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

10.9 Advanced Topic: Image Display Range

Prior to Jan 2005, image display in vgui_image_tableau was limited to 256 (vxl_byte) levels per pixel component. Images with pixel data types having a larger dynamic range were clamped to the range of [0 255]. The vgui_image_tableau interface now has the method,

 
void set_mapping(vgui_range_map_params_sptr const& rmp)

which defines how images with a dynamic range larger than one byte are to be displayed. If the pointer rmp is null, then the previous vgui display process is carried out as the default, otherwise range mapping is invoked.

For example if the image has unsigned short pixels, the intensity can be anything in the range [0 65535]. To generate a meaningful display, a range, [min max], is specified such that all pixel intensities less or equal to min are mapped to 0 and all pixel intensities greater or equal to max are mapped to 255. Intensities inside the range are mapped to the [1 254] remaining display levels according to a gamma or inversion function, as will be described below.

The parameters of the mapping are:

n_components_

The number of components in the image. A grey level image has one component, a typical RGB image has 3 components.

min_L_

The minimum range value (luminance) for a grey level image.

max_L_

The maximum range value for a grey level image.

gamma_L_

The gamma factor in the exponential mapping of image intensity.

invert_

If true, then the image display is inverted to form a negative image.

min_R_

The minimum range value for the red channel in a color image.

max_R_

The maximum range value for the red channel in a color image.

gamma_R_

The gamma for red channel mapping.

min_G_

The minimum range value for the green channel in a color image.

max_G_

The maximum range value for the green channel in a color image.

gamma_G_

The gamma for green channel mapping.

min_B_

The minimum range value for the blue channel in a color image.

max_B_

The maximum range value for the blue channel in a color image.

gamma_B_

The gamma for green channel mapping.

use_glPixelMap_

If true the range map is processed by hardware when available.

cache_mapped_pix_

Under panning and zooming operations, it is not necessary to re-map the pixel intensities. The range mapped display can be cached to avoid mapping computation by setting cache_mapped_pix_ to true.

The gamma function is defined as

 
                          1
        Ig         I     ---
       -----  =  (----) gamma
       Imax       Imax

Assume that the pixel intensity has been mapped to the range [0 1.0], e.g., I/Imax. The normalized intensity is raised to the power 1/gamma. The rationale for this definition is that a typical CRT display monitor has a non-linear response with exponential factor gamma. This correction compensates for the monitor response and achieves an overall linear intensity display.

In typical operation, the user will interactively adjust the min max values in a loop that displays the mapped image until a satisfactory display is produced. The loop should re-instantiate the parameter block on each iteration since the update is triggered by a change in the value of the rmp pointer. For example,

 
vgui_image_tableau_sptr itab = vgui_image_tableau_new();
...
//set an image on itab
...
//set up a mapping parameter block
unsigned short min_val = 10000, max_val = 40000;
float gamma = 1.0;
bool invert = false;
bool use_glPixelMap = true;
bool cache_buffer = true;

vgui_range_map_params_sptr rmp =
   new vgui_range_map_params(min_val,max_val, gamma,
                             invert, use_glPixelMap, cache_buffer);
//start range mapping
itab->set_mapping(rmp)
itab->post_redraw();

//change the range
rmp->min_val_ = 15000;
itab->post_redraw();

//the image display will be updated with the new range min value

An example of mapping is shown in Figure 13. An example of the inversion mapping for a color image is shown in Figure 14.

range-display
Figure 13: A display of an x-ray image with 16 bit unsigned short pixels. An inline tableau is used to adjust the range by moving the bars via mouse interaction. The image contrast is displayed simultaneously with the mouse motion. A histogram is also displayed to guide the user. Note the mouse position / image intensity display at the lower left indicates the pixel value in the proper units and range.

lena

Figure 14: The use of range mapping to invert the image color channels. Note again that the mouse position /pixel intensity display provides the appropriate values corresponding to the original image, not the displayed image.


[ << ] [ >> ]           [Top] [Contents] [Index] [ ? ]

This document was generated by Rumpelstiltskin on November, 22 2009 using texi2html 1.78.