Data Processing and Visualization in Java using Processing Extension in VS Code

First published on MSDN on Feb 27, 2018

Microsoft MSP Workshop

Thanks to the internet, the amount of data available for analysis has been growing exponentially for a while now. With this vast expanse of data available at our fingertips, it may be difficult to make sense of it all. This is where data visualization comes in – with the right visualization, even millions of data points can be interpreted easily.

Today, we’ll be giving an introductory talk on using Processing to create data visualizations - the skills you learn here will allow you to easily visualize any dataset and provide invaluable experience with Processing – an industry standard data visualization tool.

Programming languages can be hard to understand without clear explanation. Having been through that period, we wanted to deliver programming content at a very understandable but also fun level. For this talk, we started off with very simple explanations of core programming concepts and then moved onto more complex actions such as animations. So, without further ado, let us present ourselves.

Onur Ozturk

Hi, I’m Onur Ozturk and I’m currently a second year Information Management student at UCL. My interest in programming started with my university career and I am looking to improve myself in this field by learning new frameworks, such as Processing.

LinkedIn: https://www.linkedin.com/in/onur-ozturk-02a270133/

Kiran Gopinathan

Hi, I’m Kiran Gopinathan. An undergraduate Computer Science student at UCL. I’m interested in low level programming and have been recently looking into writing decompression algorithms in Rust.

LinkedIn: https://www.linkedin.com/in/kiran-gopinathan-5a379612b/

Some pictures from the event:

Setup – for Windows

We’ll be using Microsoft Visual Studio Code with the Processing Plugin to develop the code for this blog.

1. Install Microsoft Visual Studio Code https://code.visualstudio.com/

2. Install the Processing Plugin (Press CTRL + SHIFT + X > Type in “Processing” > Pick “Processing Language by Tobiah Zarlez”) https://github.com/TobiahZ/processing-vscode

3. Add Processing to your path.

A.From the desktop, right-click My Computer and click Properties.

B.In the System Properties window, click on the Advanced tab.

C.In the Advanced section, click the Environment Variables button.

D.Finally, in the Environment Variables window, highlight the Path variable in the Systems Variable section and click the Edit button. Add or modify the path lines with the paths you wish the computer to access. Each different directory is separated with a semicolon.

4. Create a new folder called code

5. Inside the folder place a file called code.pde

6. Open this folder with Visual Studio Code.

7. From the Command Palette, type in Processing and select ‘Create Task File’

8. Once you’ve written some code, to build the project, simply press ‘Ctrl-Shift-B’

Setup – for macOS

1. Download and install Visual Studio Code. https://code.visualstudio.com/

2. Drag Visual Studio Code.app to the Applications folder.

3. Launch VSC.

4. Add Processing to path.

5. Create a new folder called code.

6. Open this folder with Visual Studio Code.

7. Inside the folder place a file called code.pde

8. From Command Palette (CMD + SHIFT + P), type in Processing and select “Create Task File”

9. Once you’ve written some code, to build the project, simply press ‘Ctrl-Shift-B’

Introduction

Note: All code for this section can be found at

https://github.com/klinghoffer/DataVisualisation/tree/master/Code1_SimpleCircle

We’re going to start simple. Processing is a just a toolkit that makes drawing with Java easy – this isn’t saying that it’s a toy program – Processing is used for many professional visualizations.

Processing programs are called sketches – they consist of a series of commands which tell Processing how to draw to the screen. Open up the ‘code.pde’ file, and we can start to tell Processing what to draw.

The first thing we want to do is to tell Processing what size the screen should be. We can do this using the size command.

This line tells Processing that the frame should be 640 pixels by 360 pixels high. Once you’ve written that, try pressing ‘Ctrl-Shift-B’ – you should see a rectangular frame pop up.

Great. Now, let’s try drawing some shapes. Processing provides us some basic commands to draw shapes – the are as follows:

- ellipse

- rect

- triangle

Let’s try each of these out in order. First, with an ellipse:

This tells Processing to draw an ellipse centred at coordinates 150 pixels in the x-direction and 50 pixels in the y-direction, with a width of 100 pixels and a height of 120 pixels.

Keep in mind that, like many visual applications, it’s coordinate system has it’s 0,0 in the top left corner – so as I go down the page, my y value increases, as I go to the right, the x value increases.

Try building and you should see an ellipse displayed on your screen.

Let’s try a rectangle:

This tells processing to draw a rectangle with its upper left-most corner at 50 pixels along and 100 pixels down, with a width of 100 pixels and a height of 120 pixels.

Finally, let’s try a triangle:

This tells Processing to draw a triangle with corners at the points (50, 200), (100, 300), (150, 200).

Movement

The code for this section can be found at

https://github.com/klinghoffer/DataVisualisation/tree/master/Code2_Movement

Okay, cool we can draw some shapes, but they don’t really pop. Let’s try making them move.

Currently, up until now, we’ve been drawing everything once. If we want to have movement, then this isn’t enough – we’ll need to keep redrawing everything as over time they will move.

To tell Processing to do this, we need to modify the structure of our code a bit.

This code essentially defines two functions for Processing to use. The first one ‘setup’, will be run once by Processing just at the start – this is where you might put code that you only want to run once – for example, the size() command should be placed inside the setup() function, as it only makes sense to tell Processing how large the screen should be once.

The second function ‘draw’, will be run by Processing repeatedly – this is where you would put any code that you want to run multiple times – for example the commands to draw the shapes. Let’s try that:

Try running that, and you should see an ellipse that doesn’t move. Why doesn’t it move? Well, Processing is calling the draw function repeatedly, but it keeps on drawing the ellipse in the same place. How can we change this? We can add two variables to the top of the sketch to keep track of where the ellipse is.

Now, each time we draw the screen in the draw method, we can draw the circle at the x and y coordinates and change these values:

If you run this, you should see the ellipse moving, but it leaves an ugly trail behind it. What’s going on?

Processing is drawing the ellipse at a new place each time, but the previous drawings are still on the screen. To fix this, we need to tell Processing to clear the screen before drawing each time. We can do this using the ‘background’ command which fills the whole screen in a specified colour. We will use 255 which represents white to clear the screen.

Drawing with Functions

The code for this section can be found at

https://github.com/klinghoffer/DataVisualisation/tree/master/Code3_ComplexShape

Now we’ve got a moving shape, however the shape is quite simple. Consider for example, we had a house, made out of a triangle and a rectangle:

If we wanted to make the house move, we might do the same thing as earlier and draw it at two variables:

But then if we wanted to have two houses, we’d have to repeat this code using the variables for the other house – but this can easily become complex and hard to understand – just by looking at these two lines, nothing tells you that it draws a house. We can fix both problems by creating a function to draw a house:

Now to draw two houses given by positions (x1, y1) and (x2, y2) respectively, the code becomes:

Bringing it all together, our code for drawing becomes:

Now we can draw houses without having to repeat ourselves.

Drawing with Classes

The code for this section can be found at

https://github.com/klinghoffer/DataVisualisation/tree/master/Code4_ComplexShapeClass

Up until now, the code we’ve written has used the variables (x1, y1) and (x2, y2) to represent the positions of two houses. However, there’s nothing in the code that would quickly tell anyone how x1 and y1 are used. Furthermore, if we wanted to add another house, we’d have to add two extra variables, and an extra draw function call and add more lines to update the position of the new house.

We can solve these problems by using a class. A class is just a way of representing the relation between functions and data in the code. Let’s demonstrate how this works by using a class to draw Houses instead.

First, we need to tell Processing we are creating a class:

Now, inside a class definition we can put two types of statements – variable declarations like ‘int x;’ and function declarations. Any variables we declare inside of a class definition will belong to the class. So, what belongs to a House? Well, in our code, each house owns an x and y coordinate.

A class is just a template for a thing, so each instance of a class will have the variables declared in the class definition. In our case, each instance of a house will have a x and a y variable – but what values will they have? Well, we need to tell the house which values it has, when it is made. We can do that using a constructor:

Any function we place inside the class definition will define an action that the class can do – what can Houses do in our code? They can be drawn, and they can be moved – we’ll represent these using a ‘draw’ and ‘update’ method respectively.

Bringing it all together, we can edit our drawing code as follows:

Now we have the same as before, but we can add new houses easily.

Velocity-driven Movement

The code for this section can be found at

[https://github.com/klinghoffer/DataVisualisation/tree/master/Code5_ComplexMovement

Now we know how to make basic shapes move, we will be moving on to creating more complex movement patterns. Let’s start with the following base class:

Because we’ll be changing the position values a lot, from now on, we will be using vectors to represent position – a vector just allows us to change an equation of the form:

Into the form:

Processing provides us with a class that represents a mathematical vector – this class is PVector. Let’s replace all the x and y variables in our class with a PVector.

Now, our thing class moves in a constant direction – how can we change it to move in a more interesting way? Well, objects in reality, don’t just move in straight lines, they have velocity. Let’s add velocity to the shape. How can we do that? Well with another PVector for velocity.

This thing should now move in the direction of it’s velocity, but we have now way ever of changing the velocity from 0 so it doesn’t move at all. Let’s add a function to make it move towards a position:

Bringing it all together, our drawing code becomes:

Now, if you run that, the ellipse should gain velocity towards your mouse each time you click.

Accelerated Movement

The code for this section can be found at

https://github.com/klinghoffer/DataVisualisation/tree/master/Code6_AcceleratedMovement

We can make the object move more dynamically by introducing acceleration. First, let’s add an acceleration variable to the class, and change the velocity based on that value.

But if you try running that, you’ll see that it speeds up too quickly. What’s going on here is that as soon as we make acceleration a non-zero value, the velocity will keep on increasing every update leading to an exponential change in position. We can avoid this, be resetting acceleration each update. So if you apply acceleration to an object, it will change velocity for a brief moment – like if you were to apply a force.

By introducing this change, the motion of the ball has become more nuanced. While we are changing the motion of the ball, let’s also prevent it going offscreen with a custom function:

Forces

The code for this section can be found at

https://github.com/klinghoffer/DataVisualisation/tree/master/Code7_Collisions AND

https://github.com/klinghoffer/DataVisualisation/tree/master/Code7_Forces

Currently, everything we draw reacts in the same way - no matter how big or small they will all move in the same way. We can change this and add a little variety to the visualization by introducing a mass value – this will determine the size of the ball:

Now the mass of the ball changes it’s size, but the motion is independent of the size of the ball – let’s change that - instead of changing acceleration directly as before, we’ll use forces to change the motion of the ball.

Now, when you click the mouse, smaller things should move towards it faster than larger things – this makes it easier to identify large things in a crowd – this will come in handy when visualizing many data points.

Bringing it all together, our drawing function becomes:

Thing smallThing = new Thing(100, 200, 10);Thing largeThing = new Thing(100, 300, 30); void mousePressed() { smallThing.moveTowards(mouseX, mouseY); largeThing.moveTowards(mouseX, mouseY);} void draw() { smallThing.draw(); largeThing.draw(); smallThing.update(); largeThing.update();}

Collisions

The code for this section can be found at

https://github.com/klinghoffer/DataVisualisation/tree/master/Code8_Collisions

Now, the things move around and can be attracted to our mouse, but they don’t really bounce with each other. How can we make them bounce?

Well, we can split this task into two smaller subtasks – detecting a collision, and responding to a collision.

How can we detect a collision? Well the shapes in our example are very simple, so collision detection can be done by comparing how far away their centres are.

In our Thing class, let’s add a function to detect collisions, and a function to react to collisions:

We can check for collisions by checking if the distance between each thing is less than the sum of their radii (which is equal to their mass, because of how we draw them).

Now, let’s call these functions inside the draw function to make the shapes react if they collide:

Multiple Collisions

The code for this section can be found at

https://github.com/klinghoffer/DataVisualisation/tree/master/Code9_MultipleColissions

Now we have two shapes that move around and bounce off each other but if we wanted to add another point, we’d have to write extra collision code. This is very similar to earlier where if we wanted to draw multiple houses, we would have to introduce extra variables. In fact, it’s so similar, we can solve it using the same solution – introducing a class.

So now we have a class that represents a collection of things (we are using ArrayList to represent multiple things, and the for loops allow us to do an action for each Thing that is inside our list).

Now we just need to make the things collide – the place to do this is inside the update method. We can make colliding objects bounce by considering each possible pair of objects, checking if they collide, and if they do, making them bounce.

Bringing it all together, our drawing code now becomes:

Colouring

The code for this section can be found at

https://github.com/klinghoffer/DataVisualisation/tree/master/Code10_Colours

Currently, the whole screen is plain white, there’s no colours at all. Let’s fix that.

Ideally, we want the colours of the things to change based on their speed and acceleration as this will create a very dynamic scene which can easily keep viewers interested. To do this, because of how we’ve structured our code, all we need to do is edit the draw function of our Thing class.

Here, we’re calculating an energy value based on the current velocity of the object, the drawing two circles – the first one is quite large, but has a high transparency (we also use noStroke()) to remove the stroke around the circle – we then draw a smaller circle with a low transparency.

Gravity

The code for this section can be found at

https://github.com/klinghoffer/DataVisualisation/tree/master/Code11_Gravity

Our objects have mass, and they bounce off each other which makes for a pretty dynamic scene. However, we can make the visualization even more interesting by introducing gravity. First, we want to add a function to the Thing class, to allow things to attract each other:

Here we use Newton’s equation for gravity to apply a gravitational force to another object.

Finally, inside our population class’ update, as well as checking for collisions we want to make each object attract each other.

Now our visualization has become a lot more dynamic.

Data Visualization

The code for this section can be found at

https://github.com/klinghoffer/DataVisualisation/tree/master/Code12_Visualization

Now we have a pretty interesting visualization, let’s base this visualization on some data. We’ve uploaded a DataSet.pde class to GitHub – download it and add it to your project: https://github.com/klinghoffer/DataVisualisation/blob/master/DataSet.csv

To visualize the data, each time we click the mouse, we’ll just generate a new thing based on values from our dataset. To do this, change mousePressed() as follows:

Brilliant. If you now run the sketch, each time you click you should see a new Thing pop up at the location of your mouse, using the data from your dataset.

Conclusion

As you can see, data visualization in Processing can have some very enjoyable results. These results are not only useful in academics but also in professional life as Processing is capable of delivering products at an industry standard. By improving on the knowledge gained throughout the tutorial, you can also create very enjoyable animations yourself – at the expense of having fun.

We hope that you could follow along and understand the basic concepts of programming. For any questions or suggestions, do not hesitate to contact UCL Microsoft Student Partners from the following links:

Twitter: https://twitter.com/UCL_MSP

Facebook: https://www.facebook.com/UCL.MicrosoftStudentPartners/

Thank you!