An Application Within An Application

Sometimes you set out to build something and it takes on a life of its own...maybe even having a different use that you originally intended.   I've always been amazed at how software applications and languages, given a set of features and functions, can be used to build something amazing, something that the original developers never knew could be done.

They say a picture is worth a thousand words.  Nothing could be more true than trying to visualize how a network is put together.  Even walking into a lab or server room and looking at the racks and cables and systems leaves a lot to be desired.  How are my applications communicating?  What are the conversations going on in my network?  What has discovered what?

The Plexxi Control networking view needed to show this information.  Tell me about my Plexxi Switches and what they know about.  What traffic are they carrying?  Who's talking through them.  So we built a network visualizer to show the Plexxi Ring and a handful of networking device types:

We'd display our ring, the connected switches and "buckets" of machines, storage and virtual hosts/machines.  But what about routers and switches and other machines, generic machines, etc.  How can I put these on my diagram to better define my customers network?

Given our ability to display a set of objects and their connections, why not open this up to additional types of objects?  Why not allow the customer to define their own icons (maybe they use Juniper gear or Dell or Cisco or HP)?  Why not allow these new objects to be connected wherever the customer wants?  We can extend our network visualizer to do this...

And so, with just a little more code and a few more "Element" features, our basic network display tool has morphed into a full fledged network visualizer.   We can use it to build just about any network diagram.  We can get a better picture of what's connected where and how our communication is flowing.   Our network application is now a network visualization application too. 

I think I can hear the Plexxi product managers already thinking about a fancy "print" function to take a large network and stick it up on their wall.

The Most Useful Dialog I've Ever Built

For many years, I've been lucky to work on some very cool, interesting, fun, and important projects.  Sometimes, when you are fully immersed in the technology you are building, you don't always see the big picture.  Later on, you might be able to look back and fully grasp what you designed, what you built and how someone might actually do something useful with it.

I thought that my very first project, coming out of college and joining Digital Equipment (DEC), was the coolest thing.  It was called DECplan, a project planning and management system.  Remember Gantt Charts?  Precedence Diagrams?  Project Schedules?   I was tasked with "reports".  That's it.  Build a way for users to get reports out of the system.  I designed and built the Report Layout Editor.  A way to take a blank sheet of paper and customize a report.  Basically, drag database fields onto the page and then "generate" the report.  Looking back, our group built Microsoft Project before there was even a Microsoft.  I had no idea that I built a Microsoft Word like interface with one of the first graphical toolkits in the industry....DECwindows.  Very useful.

Next stop, Wellfleet.  A small networking company that had grand plans of doing big things... and it did.  As it grew, to Bay Networks and eventually Nortel Networks, I was lucky to be on the team that was tasked with managing *ALL* the networking products....routers, switches, hubs, and on.  Our lab was huge, with one of every product in the companies offering.  The interface (now using SUN equipment with Motif) had to show all the different devices, each with unique settings, fields and options, in a generic way that allowed for ease of use.  This was a challenge and a success as it did display all the equipment it discovered in a single cohesive view.  Very useful.

Virtualization was born.  An industry that has spawned numerous companies and has provided thousands of jobs to this day.  Now using Java and Swing and being able to really build not only a useful application, but something that was pleasing to use and visually intriguing, was the next challenge.  I was lucky again, to land in the fast paced startup world in the middle of an emerging technology company, Virtual Iron.  Design and build an application for managing virtual machines.  Show me what's running, allow me to rollout new machines quickly and put it all at my finger tips.  Very useful.

Along came SDN.  To be fair, SDN occurred a few years after I had already started at my next lucky stop, Plexxi Inc.  Long before network overlays, SDN, DevOps integration, and all the present day technologies started coming together, Plexxi was already working on it.

Design and build an application that shows me application conversations in my network, lets me put policies on those conversations, automatically adjust my network and show me the results.  Wow.  Of course this would be the most useful application I've ever built.   Then, one day, well into the development of Plexxi Control, the subject of DevOps and troubleshooting and network management came up.  I need to find something in my network, but I don't know much about what I'm looking for.  Maybe I know the IP Address, maybe a name, maybe a type of object, maybe a MAC Address.  Help!

And there it was, "The Most Useful Dialog I've Ever Built."

Can you find this for me?  Do you have this object in your controller database?  What's the name? Do you know the MAC or IP address?  What operations can I do to it?  Which Plexxi Switch sees this object?  What port is it seen on?  Is it on a VLAN? Is it Affinitized? 

It may seem simple, but being able to access and display this kind of information from our central controller helps answer ( in a single press of a "Find" button) a very complex question, that typically involves querying several applications, people, spreadsheets, emails, and all sorts of disorganized network information.  Very very useful.

Show Me An Affinity

- Simplifying Complex Processing with a User Interface -

Every application I have ever worked on had the same end goal; provide useful, simple, easy to use and visually appealing screens.  Don't confuse me.  Don't make me click hundreds of times.  Don't leave important information out.  If something fails, tell me what to do next.

In designing a set of views and screens for my current project, the problem statement can be summarized as follows:

• Show me where my applications are in my network
• Let me pick the important applications and designate them as such
• Let me define how these applications should be conversing
• Do some SDN magic on these applications
• Show me where my applications are in my ring

Providing the ability to see and do all of these actions requires a full application suite where common information is displayed from different perspectives.

In my Network view, I can search for specific virtual machines, among other things, and put them into an affinity group, thereby capturing the importance of an application in my network.

I am interested in how my three applications:

• authhost71
• authbox81
• authclient62

converse with their backend databases dbserver28 and dbserver84. 

Everything in App1 talks with DB1, bidirectionally,  and I'd really like these conversations to be isolated from other ring traffic.

Is my ring ready for the SDN magic fitting algorithm to do it's thing?    Looks formed, healthy, good to go.

For each of my conversations, where are my applications traversing the ring?  My fitting results shows me how each application in App1 talks with each application in DB1; where it enters my ring, how it traverses my ring and where it exits my ring.

I found some applications in my network and after affinitizing them ("is that word? affinitizing?  it will be soon...."), I see clearly how my ring is handling their conversations and isolating them from all others. 

Taking a complex set of data and simplifying the ability to work with that data is what a good user interface is all about.   Show me an affinity? I saw an affinity today.

Fun with JIT, part2

Here's some more neat things I've done with my GWT JIT integration.  The code snippets here are to represent what is possible and your mileage may vary.

Dragging Objects

In many of the JIT renderings, you can drag objects around the screen and can have full control over what you allow to be dragged.  You start by getting a callback when dragging is occurring:

onDragMove : function(node, eventInfo, e) {

where the "node" is the object that is being dragged and "eventInfo" has position information.  For a Force Directed JIT graph type, to allow dragging of your objects, you simply do this inside the onDragMove function:

var pos = eventInfo.getPos();
node.pos.setc(pos.x, pos.y);
fd.plot();

By the way, "fd" is an instance of the Force Directed graph.

As you drag the object around the screen, it will be moved and the graph will be replotted to show the update.  If the object you are dragging has any edges attached to it, they will automatically be moved as well.  But what if you have relationships on your display?  What if you have a parent object that has several children connected to it and when you drag the parent, you want the children to be dragged as well.  At Plexxi, we display a RING object with Plexxi Switches connected to the ring.  If a user drags our ring object, we want the switches to go with it.  Here's how:

onDragMove : function(node, eventInfo, e) {
    var pos = eventInfo.getPos();
    if (node.getData('type') == "plexxiring") {
        node.eachAdjacency(function(a) {
            childnode = fd.graph.getNode(a.nodeTo.id);
            childnode.pos.setc(childnode.getPos().x - (node.getPos().x - pos.x),
                                          childnode.getPos().y - (node.getPos().y - pos.y));
        });

    node.pos.setc(pos.x, pos.y);
    fd.plot();

What we do is get all the nodes connected to our dragged node and move them as well.  This performs surprisingly well in our application.  You can take this further and re-curse through all your objects moving even more when a single one is dragged.

Mouseover Tool Tips

On some occasions, you may want a tooltip or a text box to show up on the screen as the user moves the mouse over your nodes.  To insure good performance, you will want to add the tooltip text to a private JSON data object so that this text is instantly available to you at any time.  If you have to go back to your database of make some other calls to get your tooltip text, you may suffer from bad performance in rendering them. 

Here's how I did Tips with my Force Directed graph:

Tips : {
    enable : true,
    type : 'Native',
    offsetX : 10,
    offsetY : 10,
    onShow : function(tip, node) {
        tip.innerHTML = node.getData('tooltip');
   },

Auto Refreshing

One of the biggest issues with any graphical application is refreshing.  Does your user have to press a refresh button?  Does that waste time retrieving data from your database, destroying whatever you are currently displaying and rebuilding the display with new data? 

By using GWT and the built-in RPC mechanism, I am able to ask my GWT Server if any data has changed and then act of it.  This is a topic for another blog....but let's say my Force Directed graph is showing 5 different types of objects on it.  I ask the GWT Server to coordinate with my backend database to keep a cache of the objects I'm displaying and if any of these objects is modified, or deleted, let me know.  When I find out something that I am displaying has changed, then I take action.  The most common events are creating or deleting a link (edge), deleting an object or changing the name of an object.  Here's how they are handled as called by my GWT code:

private native void createLink(JavaScriptObject nativeGraph, String link, String from, String to)/*-{
        if (nativeGraph.graph.getAdjacence(from, to) == null) {
            var fromNode = nativeGraph.graph.getNode(from);
            var toNode = nativeGraph.graph.getNode(to);
            var adj = nativeGraph.graph.addAdjacence(fromNode, toNode,
                            {
                                'linkUuid' : link,
                                'selected' : 'false',
                                'linkcolor' : '#FF0000'
                            });
              
        nativeGraph.plot();
    }
}-*/;

private native void deleteLink(JavaScriptObject nativeGraph, String linkUuid)/*-{
        nativeGraph.graph.eachNode(function(n) {
            n.eachAdjacency(function(a) {
                if (a.data.linkUuid == linkUuid) {
                    a.setData('alpha', '0');
                    nativeGraph.graph.removeAdjacence(a.nodeFrom.id,
                            a.nodeTo.id);

                    nativeGraph.plot();
                    return;
                }
            });
        });
    }-*/;

private native void deleteNode(JavaScriptObject nativeGraph, String nodeUuid)/*-{
        nativeGraph.graph.eachNode(function(n) {
            if (n.id == nodeUuid) {
                n.setData('alpha', 0, 'end');
               nativeGraph.removeNode(nodeUuid, {
                     type: 'fade:seq', 
                    duration: 1000, 
                    hideLabels: false,   
                    transition: $jit.Trans.Quart.easeOut   
               });
            }
        });
 }-*/;


private native void updateNodeName(JavaScriptObject nativeGraph, String uuid, String newName)/*-{
        var node = nativeGraph.graph.getNode(uuid);
        if (node != null) {
            node.name = newName;
            nativeGraph.plot();
        }
}-*/;

fun with JIT part3 might be in the works......enjoy.

Fun with JIT, part1

Over the past year, I've been integrating JIT  (thejit.org) with my GWT application.  Javascript is not something for the faint of heart and personally I try to stay far away from it instead allowing  GWT to generate the Javascript for me based on my Java implementation code.  But to make use of, extend, and better control the canvas based layout tool JIT, I had to take on some Javascript.  So here's some things I did with it.  Any code snippets here should be considered pseudo-code and are for representation only.

Saving & Restoring Node Positions

The Force-Directed Graph has a built in layout algorithm that draws a graph as best as it can.  However, each time the graph is rendered, the layout algorithm produces a different layout.  We offer the user the ability to control the layout somewhat, specifying spacing between nodes and how many iterations the algorithm should use based on a complexity setting and some others:

Once the layout has been run, we then let the user drag the nodes and links around the screen to better position items to their liking.  To fully make this work, we need to save the node positions so that the next time we view the graph, we put all the nodes back into their original positions.  This also provides a big performance boost, in that we never run the layout algorithm a second time, but instead tell JIT where we want the nodes.

Save Positions:

// We store the node positions using our string object id, and the x,y values.

HashMap<String, ArrayList<Integer>> nodePositions;

// We iterate over the graph, getting the positions of each node.

mygraph.graph.eachNode(function(n) {

saveNodeData(n.id, Math.floor(n.getPos().x), Math.floor(n.getPos().y));

});

saveNodeData(String id, int x, int y)
{
    ArrayList<Integer> locations = new ArrayList<Integer>();
    locations.add(x);
    locations.add(y);
    nodePositions.put(id, locations);
}

Restore Positions:

// Lood our JSON data

nativeGraph.loadJSON(data);

// Put the nodes at the positions we want. No need to run the layout algorithm.

iterate over the saved node positions data and call this:

setNodePosition(JavaScriptObject nativeGraph, String id, int x, int y)
{
    node = nativeGraph.graph.getNode(id);
    if (node != null) {
        node.setPos(new $wnd.$jit.Complex(x,y), 'current');
        nativeGraph.plot();
    }
}

Resizing the Graph

JIT comes with some built-in functionality on many of their graphs that allows the user to scroll their mouse wheel which then causes the canvas to scale in and out, making the drawn objects grow or shrink.  I found this functionality to be somewhat difficult to control and also I wanted to control how large and how small I would allow the screen to scale to.  

I implemented two push buttons on my graph that allow the user to scale up or down as they are pressed.  One jumps 125% and the other jumps 80% which provides an even up/down scaling so that they user can always return to the default, 100%.

Scaling In:

Scaling in makes the objects larger.  Each time the user presses the "+" button, I ask JIT to:

nativeGraph.canvas.scale(1.25, 1.25);

Scaling Out:

Scaling out makes the objects smaller.  Each time the user presses the "-" button, I ask JIT to:

nativeGraph.canvas.scale(.8, .8);

Searching the Graph

Since my graph can get quite large with many of the objects off the edges of the viewable area, I implemented the ability to search for any node by name.  Once found, the node would be selected and the graph would be centered on the object, bringing the object into view for the user.  Here's how.

Find Node:

findNode(JavaScriptObject nativeGraph, String name)

{

    var p = new RegExp(name.replace("*", ".*"));

    

    nativeGraph.graph.eachNode(function(node) {

        if (p.test(node.name)) {

            // Found the node, we can get the x,y positions of this node.

    centerScreen(node.getPos().x, node.getPos().y);

}

}

centerScreen(String x, String y)

{

    // We are using a scroll panel with scroll bars as the parent of our JIT canvas

    // so we just adjust these scrollbars.

    graphContainer.setScrollLeft(((graphWidth / 2) -
(getOffsetWidth() / 2)) + x);
    graphContainer.setScrollTop(((graphHeight / 2) -
(getOffsetHeight() / 2) + y);

}

...More Fun with JIT, part2 coming soon.

All Artists Need A Canvas

In designing user interfaces, the widgets provide the starting point.  The widget set typically includes lists, tables, trees, buttons, menus, and dialogs, and provides the interface designer with his tools.   Mapping data into a tree or a table, styling the widgets for a branded look and feel, adding scrolling, sorting, and selection behavior brings a user interface to life and provides the end user with instant visibility.

But sometimes these basic widgets don't go far enough with their ability to convey real world concepts. If a picture is truly worth a thousand words, a graphical four column table with rows of data does not a picture make.  So what alternatives are there?  What if I need to display objects arranged in a circular fashion with relationships between them?  What if I need to show a grab bag of elements that share various connections with each other?  How do I give the user that one picture worth a thousand words?

Given my requirements, I went searching for my etch-a-sketch that would both allow me to arrange objects and at the same time help me with the organization of them.  Enter JIT - Javascript InfoVis Toolkit.  A compact, simple, extendable and powerful canvas drawing and manipulation tool that could be easily integrated into my GWT-based application that was already underway.

In conjunction with the JIT documentation, examples and experts on the Google-groups, I was able to build a view that would provide our users with a visual rendition of network components and their connections between each other as shown.

Network View

We achieved support for:

• Search by name with the resulting object being selected and centered on the screen
• User draggable elements for user positioning or JIT auto layout algorithm activation
• Saving of all element positions to redraw later using the same layout 
• Zoom in and Zoom out functions for canvas scaling controlled with a "+" and "-" button press
• Spacing between elements, canvas size, font size and name truncation controlled with view options
• Full element selection, extended selection and right click popups for shortcut operations
• Full link (or edge) selection, extended selection and right click popups for operations
• Dynamic view refreshing using "sum" and "morph" JIT technologies via JSON data
• Much more to come....

What about being able to show a set of objects related to each other in both directions?  A set of elements that form a ring?  Something that supports selection and can show elaborate connections, but that auto layouts the diagram perfectly each time?

Ring View

With the sunburst layout we achieved support for:

• Search by name with the resulting object being selected on the screen 
• Zoom in and Zoom out functions for canvas scaling controlled with a "+" and "-" button press
• Ring size and spacing, font size and name truncation controlled with view options
• Full element selection and right click popups for shortcut operations
• Link (edge) highlighting using various colors based on search or selection criteria
• Dynamic view refreshing
• Much more to come....

By combining the asynchronous processing power and speed of GWT with a simple javascript canvas drawing and manipulation tool, we are well on our way to building an interface that fills trees, tables, buttons, menus and dialogs and provides that thousand word picture.

You Can Go Home Again

Who says you can't go home again?

As a newly graduated computer scientist, my own backyard was full of companies to go to work for - Data General, Wang, DEC, Apollo, and others.  These companies were booming and were gobbling up my classmates.  Living in New Hampshire, I chose DEC and landed at the Spit Brook road facility that included the famous ZK1, ZK2 and ZK3 buildings.

I traveled to California, Vegas, even Valbonne France as Digital grew and grew.  I sat next to seasoned consulting engineers that knew how to build fantastic software.  I learned.  I saw the creation of computer graphics via DECWindows based on the X Windows framework and later to Motif.  I was hooked with UI design.  I originally worked on DECPlan (basically Microsoft Project before there was a Microsoft) and to this day, it had features that no project management system has.

Mid-90's....the downturn starts.  Competition heats up, the PC is born, Wang starts to collapse, DEC has layoffs and salary freezes, change is brewing.  After 10 years, it's time to move on.  There's some new up and coming companies called Cabletron, Wellfleet, Cisco out there that are the next big things.

And so, some 25 years after beginning in ZK3, I have returned to Spit Brook road.  It's no longer one company...the buildings have been updated, carved up to house numerous companies.  I recently took a walk to some of the uninhabited floors and areas.  I found a conference room with some very interesting information printed on the wall:

The list of VMS features spanned from BL5 (which boasted 100 customers) all the way to version 5.0. It was almost eerie.....like the engineers had just packed up and left and now only dust was living here.

The buildings have also undergone some cosmetic changes as well.  All the entrances have been redone, the heated outside tile gone, and a more engineered entrance in place.

The interior hallways have new carpet, and a new paint scheme with more modern lighting.

A piece of DEC history has also been restored.  Many old timers will remember the barcode on the entrance to ZK2, where the main entrance and credit union used to be.  It spelled out Digital  Software Engineering at some point and has long been removed.  But this past summer, this showed up on the cafeteria wall:

With a plaque that reminded everyone of the barcode DEC used to have and that this new barcode, in the spirit of the original one, spells out Nashua Tech Park.

In my travels about the buildings, I found a three ring binder with a Digital logo on it.  I've put some of my Plexxi documents into it.  There's good karma in these buildings.  Maybe the next big thing is coming from this historic location.  You CAN go home again.