Final Project

In this final project you will be bringing the conceptual dimension of the class together with the visualization techniques we’ve learned. You will develop and implement a final project following a complete, iterative design process. The first step in this is the creation of a set of proposals.

Process

You should aim to generate about 10 ideas during the next week that are intellectually distinct, address a diverse range of points of view, and explore different subject domains and levels of complexity. Work rapidly and don't get hung up on polishing anything digitally — limit yourself to handwritten notes and sketches during this phase while you are doing research (recommended: 10 ideas in a 1 hour session).

During the research and sketching process, address the following questions:

• What are some observations you find noteworthy?
• What insights could this offer?
• How is it related to visualization, what can visualization reveal here?
• What new ideas came out of the process of sketching and research?
• What are the next steps?

Refine your list daily and let your initial set of ideas sit for at least a day before you take things a step further. Talk about your ideas with a peer outside of class (their level of expertise does not matter here).

Ideas & Sources (due 14 Apr)

List your ten ideas in the process/ideas.md file in your 4.final-project directory. Each idea can be described in 2 or 3 sentences, but make sure you outline the subject matter being considered, the question being asked, and the format (digital vs print, static vs interactive, diagram vs text vs map, etc.) you intend to use.

For five of these, find at least one datasource and record the details in your process/datasources.md file. There is a placeholder entry in the file describing the USGS feeds. Delete that entry, but use it as a model for the five new entries you'll be adding.

Concept Design & Development

Based on the feedback you’ve received for your 10 ideas from the ideation phase, choose one direction and create three different series of wireframe sketches (with at least 2 sketches per series) to illustrate how you plan to convey your subject visually. Your wireframes will be static representations of the eventual user interface plus schematic views of your visualization. The individual components of the sketch should be rough, functional approximations of the final placement, size, and interaction type (click, drag, hover, etc.) of UI elements but shouldn't yet be concerned with aesthetics.

Develop these sketches quickly enough to explore alternative approaches that present the subject in different ways. Experiment both with the visual representation of the data (in terms of the retinal variable mappings we've examined) and the different affordances your UI provides, allowing the user to pose different questions and filter/sort/focus the information in different ways.

In parallel with your design work, build a proof-of-concept illustration that your data-source is sound and will be able to provide the quantitative and qualitative information necessary for your final visualization. To that end, write a simple p5 script in which you read in one of your data files (whether CSV, JSON, or otherwise) and print/draw its values to a canvas. This sketch will be the basis for the visualization portion of your project and should not incorporate any of your UI design ideas until we get to the next step of the process.

Sketches and Data (due 21 Apr)

Select one idea to develop further for your final project and create a one-page proposal, 3 wireframe sketches, and a p5 script that loads in the data you’ve collected as either a JSON file or CSV table:

• Summarize your final project ideas as a 1-page project proposal. Include rough visual sketches to illustrate your approach as needed.
  • The written description can be short (200–500 words) and should establish the subject matter, the question you’re trying to answer, and what you expect your intended audience to take away from the project. Be sure to address the questions listed above in the “Process” section.
  • The sketches are there only to help communicate your overall concept, so don’t spend too much time ‘designing’ them. If you don’t include a sketch, then describe your visualization approach thoroughly.
  • Create a PDF called proposal.pdf that combines your written description and supporting sketches and place it in the process folder.
• Bring in three conceptually distinct approaches to visualizing your chosen dataset, depicted with at least 2 sketches apiece.
  • The sketches should illustrate both the representation of the data and the user interface approach.
• Collect the data files (plural!) that contribute evidence to your subject (look in particular for CSV, TSV, JSON, XML, or SQL sources) and write a p5.js script that reads in your data file and generates something visual from it.
  • The form of the visual representation is not important, but it must include every field from the data source that you'll be using in your final visualization.

Initial Prototype (due 28 Apr)

Continue developing your wireframes and draw up a set of static representations of your user interface that demonstrate all of the ‘feature complete’ version’s eventual affordances, modes, and states. Devote particular attention to screen (or page) layout, typography, color mappings, contrast (both in terms of lightness and type hierarchy), and overall look and feel. These static mock-ups will serve as the ‘specification’ which the rest of your development work this semester will attempt to implement.

• Collect your mock-ups in a single PDF file and place it in your repository as process/prototype.pdf.

• Clean your data and convert it to a more programmatically usable (CSV, JSON, TXT, or XML) form. If you’re working off of a ‘live’ data source, learn the relevant API. In the case of relational databases, ensure you have a full understanding of its schema and prepare the queries you’ll be using. Submit your p5 sketch that successfully parses the data as data/sketch.js and place any local data files you’ll be using in data/assets.

Final (due 12 May)

You will present your completed projects at our last class meeting. Good luck!

Exercise 3: Mapping Quantities, Categories, and Summarized Data

For this third project, we'll be examining a simple time-series dataset: the history of nuclear testing by the eight (declared) nuclear nations. In the first phase of this project we will consider only the total number of test explosions across three dimensions:

1. the state conducting the test
2. the year in which it occurred
3. whether it was above- or below-ground

Despite the simplicity of the data, there are a number of ways to slice and group the data. Some things to consider:

• Is each year plotted separately, or are the totals grouped into decades or some other multi-year chunk
• Is time represented by position on the x or y axis? Or by angle? Or using color?
• What are the 'objects' being represented in your diagram? Are they the countries, the years, or the individual tests?
• How will your diagram address the 'lull' that resulted from the Comprehensive Test Ban Treaty in ’96 and the more recent resurgence in testing?

Preliminaries

Starting with this project, we'll be making use of external files to separate the data we're plotting from the code that does the actual drawing. Getting this to work means becoming acquainted with a couple implementational details of the browser environment.

The first is the idea that retrieving information over the network is not instantaneous and that some time will pass between when you ‘request’ your data file and the moment where its contents are available to your script. To work around this, p5 allows you to define a function called preload which will run before your setup or draw functions begin. In our case, that function will be used to call either loadTable (which fetches a CSV file) or loadJSON (for, unsurprisingly, JSON data). In either case, you'll want to make sure you follow a procedure along the lines of:

var externalData; // declare a global variable to store the data

function preload() {
  // start fetching the data and assign it to your global
  externalData = loadJSON('http://example.com/some/file.json');
}

function setup(){
    // ... use the global externalData variable here (or in draw)
}

The second browser-related issue is the Same-origin Policy which requires that all the files your page touches come from the same domain. Thus far, we've been opening our index.html files directly (which gives them a URL beginning with file:///). Unfortunately, this is incompatible with loading external data files, so we’ll need to run a local web server (giving everything an http://localhost/ URL).

Open a terminal window in your main di-2020 folder via the Github Desktop app’s Repository → Open in Terminal menu. Once there, type make server. You'll then see a message along the lines of:

Starting up http-server, serving .
Available on:
  http://localhost:8080
Hit CTRL-C to stop the server

Now try pasting that http://localhost:8080 url into your browser, and then start clicking on folders to navigate to your student directory. Once you're done working, you can type Control-c in the terminal window to shut down the server (or just close its window and click the Terminate button).

Getting started

Begin your investigation with this simplified dataset collected from the Natural Resources Defense Council's pages on the original and subcontinental nuclear powers.

Examine the data subdirectory of the project folder which contains JSON & CSV versions of the table below. Then look at the scripts in the examples folder. These scripts build on our work learning the p5 drawing primitives in three major ways:

1. setting drawing parameters with structured, tabular data rather than hard-coded values
2. working with serialized JSON and CSV data
3. using loops to render multiple ‘instances’ (a.k.a. rows) of a data set using the same procedure each time

Start by modifying the code in the example scripts and explore some of the different ways to represent the country, year, and test-count values. Make 'snapshot' copies of your script as you modify it so you can build up an array of different approaches.

Next steps

Over the next week, continue working with this dataset and sketch out three different approaches to representing it in a way that tells a story.

Confine yourself to static, non-interactive representations for now, but consider any medium that could represent these values; posters, projections, and physical representations are all just as valid as pixels on a screen.

In at least one of your three variations, bring in one additional variable using data you find on the web and be sure to cite your source. This additional variable could be anything from a timeline of world events to the military budgets of the countries (or even the astrological signs of their leaders). Just make sure you can defend how it is in some way adding useful context to the data.

Feel free to use whatever tools you're comfortable with. Programmatic drawing can be very handy but so can Excel…

Data sources

This spreadsheet on Google Docs has a tidied up copy of the NRDC data and provides both the total number of tests as well as distinguishing between 'atmospheric' or 'underground' tests. Consider making use of a pivot table to modify/summarize the information, and then export it as a csv for your own usage.

Also consider taking a look at the Wikipedia page listing the various tests and providing links to country-specific pages with background information about the individual tests.

For the details of individual tests, take a look at the materials collected at “Johnston's Archive” (and also included in the last tabs of the Google Sheet). But keep in mind that the volume of data is substantially larger, so be sure to have a clear idea of what you're trying to collect before diving in...

Supplemental materials

I've assembled a cleaned-up csv file with most of the Johnston's Archive data. If you're looking for information about the naming schemes of the individual tests, yield estimates, locations, dates, etc. it may be helpful...

Lawrence Livermore National Laboratory recently released the declassified 2400 FPS films of many of their atmospheric nuclear tests. Those of you whose projects are dealing with specific tests rather than the summary statistics may find stills from the videos useful in your work. Take a look at their playlist on YouTube.

Mapping Time

Preliminaries

Gather all the necessary software and files to get started:

• The Sublime Text 3 (or comparable) text editor
• The GitHub Desktop GUI client
• Create your own fork of https://github.com/samizdatco/di-2020

The P5.js site has an extensive Reference section with a full listing of the drawing commands that make up its API. It’s also got a somewhat sparser set of Tutorials that might help you get started (but mostly seem to be written to an audience that’s familiar with the original Processing environment so YMMV).

For this assignment, make sure you’ve got a handle on these basics:

• The coordinate system
• Shape primitives: rect()/ellipse()/arc()/
  line()
• Setting colors: fill()/stroke()/background()
• Mixing colors: color()/lerpColor()
• Iteration: for(…;…;…)/[].forEach(…)/while(…)

Your project files are also set up to allow you to use a pair of exceedingly useful libraries (though don’t feel like you need to learn them immediately). Take a look at their documentation pages and see if anything jumps out at you:

• lodash – the unnoficial 'standard library' of javascript with utility functions for every use-case under the sun
• chroma – powerful color-math functions useful for interpolating between colors, applying hue/saturation/brightness transformations, etc.

To get yourself situated, try looking over the sample code in the examples directory. Note the basic form of each of the programs (i.e., their setup() and draw() definitions), how they employ the drawing commands listed above, and particularly their use of variables to hold partial computations and for-loops to encapsulate repeated procedures.

Dataset

For this project the ‘data’ you'll be working with are the numerical aspects of time. You should familiarize yourself with the clock() function I've provided to help access the current date & time. The function returns a time Object with the current instant broken down into individual components. You can get numerical values for hour, min, sec, and ms on the time side of things and year, season, month, moon, day, and weekday if you’re interested in dates.

For instance, you can use the current year to work backwards:

var now = clock()
var age = 64
var born = now.year - age
print(`A ${age}-year-old would have been born in ${born}`)

To determine what half of the day it currently is, access the am and pm attributes, each of which contains a Boolean. You could conceivably use this to set the background color depending on whether it’s before- or afternoon.

function draw(){
  var now = clock()
  if (now.pm){
    background(0)
  }else{
    background(255)
  }
}

The object returned by clock() also provides a mechanism for measuring the current time in terms of the cyclic features of the calendar. By accessing fields of its progress attribute, you can get the fraction of various periods that have elapsed before the current moment. The clock represents progress as a number between 0 and 1 for each of: year, season, moon, month, week, day, halfday, hour, min, and sec.

For instance, if you wanted to be fairly literal and draw a progress bar for the day, try drawing a black rectangle whose width is proportional to the ‘doneness’ of the current 24-hour cycle:

function draw(){
  background(255)
  noStroke()
  fill(0)

  var now = clock()
  var pct = now.progress.day
  rect(0,0, width*pct, height)
}

The full set of attributes you can access via the clock function is as follows:

var now = clock()

// numerical values for elements of current time
now.hours // hour in 0–23 'military' time
now.hour  // hour in 1–12 'am/pm' time
now.min   // minute
now.sec   // seconds
now.ms    // milliseconds
now.am    // true for hours 0-11
now.pm    // true for hours 12-23

// numerical values for elements of current date
now.year    // the full 4-digit year
now.month   // month number 1–12
now.moon    // the fullness of the moon 0–1.0
now.day     // the day 1–{28,29,30,31}
now.weekday // the day of the week 1-7
now.season  // the current season 1-4 (starting with spring)

// a string-based representation that can be used as an argument to clockStart
now.timestamp // "2001/12/31 23:45:56"

// values between 0.0 and 1.0 measuring the current time's %-completion of various cycles
now.progress.year
now.progress.season
now.progress.month
now.progress.moon
now.progress.week
now.progress.day
now.progress.halfday
now.progress.hour
now.progress.min
now.progress.sec

// string versions of the date & time (in case you want to print it out)
now.text.time    // "11:45:56 P.M."
now.text.hour    // "11"
now.text.hours   // "23"
now.text.min     // "45"
now.text.sec     // "56"
now.text.ampm    // "P.M."
now.text.date    // "31 Dec 2001"
now.text.year    // "2001"
now.text.season  // "Winter"
now.text.month   // "December"
now.text.mon     // "Dec"
now.text.day     // "31"
now.text.weekday // "Monday"

Goal

• Create a visual representation of the current local time using only graphics primitives, symbols, and formal elements like color/texture/size — no text or alphanumeric characters!
• Use the clock() function so that your sketch is always displaying the current time in your draw() function.
• This is to be a creative interpretation of the idea of a clock. Your focus should be on inventiveness and polished visuals.
• Your interpretation needs to be legible to you (the author), and you need to be able to explain how the time is derived visually at any given point.
• Your clock concept needs a name (put that in the <title> tag of the index.html file).
• We will be working in two phases
  1. Representing the current hour, minute, and second by mapping their numerical values onto form
  2. Representing longer-term ‘calendar time’ units such as days, weeks, months, moon phases, seasons, years, etc.
• In each case your task is to develop an encoding of either the numerical unit itself (e.g., the number 11 if it’s 11 A.M.) or its proportional relationship to the passage of time (0.42 since 11 a.m. is 42% of the way through the course of the day) and a visual representation that makes use of what we’ve learned about Bertin’s retinal variables

Process

• Start off by making some hand-drawn (or mocked up in a drawing app) sketches of clock ideas and put these images into the process folder of your subdirectory.
• Update the README.md file in your process folder describing what’s in your various sketch files, the logic of the representation, etc. Be sure to explain, either in words or diagrammatically, your chosen mapping of time components to Bertin-ian Retinal Variables.
• Start coding by editing the project/sketch.js file. If you want to explore multiple options in parallel, use the Finder’s Duplicate command to clone the whole project directory and give it a sensible name (i.e., not just project copy or project2). You can then start working on the sketch.js in the new folder without disturbing your previous work.
• Commit your changes whenever you’ve made some modifications that feel like they’re in a stable state (or if you stumble onto a glitchy visual that points in a direction you’d like to explore). This will let you ‘rewind’ to that point in the future if you’ve hit a dead-end or need to remember how you did something.
• When you make a commit, select just the files (or even just the lines within one) that are part of the ‘conceptual unit’ of change that you made and type a brief description of what changed into the Summary field.
• Sync your local repository with the copy on GitHub before the start of class next week.