Representing Data

Tricky Data

• Human readable data is not always the best machine-readable data!

• Storing the above in a 2D array would work, but would be frustrating to work with

A Computer Friendly Approach

• Student grades are time ordered, so we could use a list for the grades
• Each student has a corresponding sequence of grades (and students are unordered), so we could use a dictionary where student names are the keys and the list of grades the values
• Each class corresponds to an unordered set of students. Could have another dictionary where the keys were the class names and the values were the dictionary of students/grades

Example Representation

{
    "Python": {
        "Sally": ["A", "B", "B", "A"],
        "Jake": ["B", "B", "B", "C"],
        "Ben": ["C", "B", "B", "A"]
    },
    "Astro": {
        "James": ["B", "B", "A"],
        "Lily": ["A", "A", "B"]
    }
}

Compound Structure Storage

• Structures representing complicated data can often be large enough that you don’t want to store them within your program itself
• We can put them in their own file, but reading them in with our current tools would be complicated
  • Current methods read in text, so we would need to parse the text to identify what data structures we needed to create and what elements we needed to add
  • This is certainly possible, but potentially more overhead than what we would like for some structures
• Useful then to store the data structure in file in such a format that it can be easily read into Python

File I/O

• A variety of ways this can be done
  • XML, YAML, JSON
• JSON is particularly interesting to us, because its syntax almost exactly matches Python’s (even though it was made for Javascript)
• Python has a built-in library to read and write JSON files, just called json
  • json.load(|||file handle|||)
    • Loads the JSON data structure from the specified file into its Python equivalent
  • json.dump(|||data_object|||, |||file handle|||)
    • Writes a JSON text representation of the data object to the given file
  • Both methods are used inside our normal with open(|||filename|||) as |||fhandle|||: syntax

Using JSON

• To read a JSON file into a variable data:

  import json
  with open('file.json') as fh:
      data = json.load(fh)

• To write a variable with complex structure out to a JSON file:

  import json
  with open('file.json', 'w') as fh:
      json.dump(data, fh)

The Power of JSON

• One very nice aspect of JSON is that it is often the defacto way that information is passed around the internet
• This means it can be easy to find data providers where you can access or download information already in a JSON format
• DND Fireball Spell info here
• We could download this information to a file, which we could then read in and use within our Python program
• Later we’ll also look at how we could process the information straight from the internet as well

JSON Gotchas

• If you are writing JSON files from within Python or using files gotten elsewhere, they should already be properly formatted
• If you need/want to edit a JSON file directly though, you should be aware of a few “gotchas” where the JSON syntax varies slightly from Python’s syntax
  • You can not have trailing commas at the end of a JSON structure
    • Something like [1, 2, 3,] is perfectly fine in Python, but illegal in JSON
  • JSON strings require double quotes
    • In Python you can use either double or single quotes, but JSON requires double
  • Booleans are all lowercase in JSON
    • Vs starting with a capital letter in Python
  • All JSON keys are strings

Compound Strategies

• If working with data that is already heavily nested, you must stop and really understand the data structure before starting to write code
  • What parts might you be needing to loop over?
  • What parts might you want to extract for later use?
• Break out smaller pieces that you are going to need to reuse into their own variables
  • This stops you from having to re-index everything out every single time
• Pay close attention to what data type you get back from each indexing
  • Is it a list? A dictionary? A set?

Demo

• We saw an example of the nested structure holding information about a fireball spell already
• I’ve gone and grabbed all 300+ D&D spells into a single data structure, where each spell has a similar internal structure to Fireball
• Let’s use that structure to write some code to do the following:
  • Print out all spells from a chosen school below a certain level threshold
  • Print out all spells from a chosen class that have no verbal component
  • Determine which spell does the most cold damage at level 11