Module #09

In this module we'll talk about how to prevent our programs from crashing when exceptional situations occur. We'll also learn about getting input from a device other than the terminal such as a local file stored on the computer or a file stored on a website.

Note that many of the topics presented in this module will not run correctly using our web-based coding environment. To try any of the code samples included in this module, please launch IDLE on your own computer and run the code locally.


Basic Exception Handling

An "exception" can be described as an error condition that causes a program to halt while it is running. In IDLE it is the angry-looking red message that appears when you encounter a runtime error. Here’s an example of some code that would cause an exception to occur in your program (you can’t convert the string “Craig” to an integer):

x = "Craig"
y = int(x)

And here is the error message that occurs when this code is run:

Traceback (most recent call last):
  File "C:\demo.py", line 2, in 
    y = int(x)
ValueError: invalid literal for int() with base 10: 'Craig'

This information (called a "traceback") includes the line number that caused the issue and the type of exception that occurred. When this happens, programmers generally say that an exception has been “raised” by the program.  Many times you can avoid exceptions all together by simply adjusting how we code our algorithms. For example, the following code will raise an exception if x = 0:

x = int(input("give me a number")) 
print (100/x)

Yet we can avoid the issue by wrapping the potentially dangerous code inside a selection statement.  For example:

x = int(input(‘give me a number’))
if x != 0:
	print (100/x)

Python has an "exception handling" statement that can be used to “catch” exceptions and prevent them from crashing your program. Here's what it looks like:

try:
	# questionable code goes here
except:
	# this block will run if the code in the
	# ‘try’ block above raised an exception 
else:
	# this block will run if the code in the
	# ‘try’ block above was successful

This should look a bit like an if / elif / else statement. When Python encounters the try statement it attempts to run the commands inside of the try block. If any of those statements cause an exception to be raised it will immediately stop executing the commands inside of the try block and jump down to the except block. Any commands listed in this block will be executed at this point. If Python does not encounter any exceptions in the try block then it will jump down and execute the statements in the else block. Note that the else block is optional - you can leave it off if you find that you don't need it in your program.

We often use try blocks to attempt "risky" tasks, such as converting strings into numeric data types or opening up files from the Internet. For example, here's a program that uses the try / except / else suite of statements to validate the data type of inputted data. Notice how you can enter ANYTHING as input and the program won't crash!

Sample program: This program demonstrates how the try / except / else blocks work in Python



Reading Data from the Web

Splitting Strings into Lists

Sometimes you are given access to a single string that contains multiple smaller values. For example, consider the following string:

student = "Tracy,NYU,Sophomore"

This string contains three pieces of data – a name, a school and a class year. The data has been "packaged" into a single string in a meaningful way (i.e. we know that the comma character separates the different data points)

We can use a technique called "splitting" to"unpack" a string and extract its individual values. The trick is to isolate a "separator" character that can be used to delimit the data you want to extract. Once you have identified the separator you can use the split method to cut apart the string into a list of values. Here’s an example:

student = "Tracy,NYU,Sophomore"

# cut apart the string based on the position of the commas
splitdata = student.split(",")

# we now have a list with 3 elements (the commas have been removed)
print (splitdata)

# we can use index notation to access the individual elements of our data
print ("Student name:", splitdata[0]) 
print ("School:", splitdata[1]) 
print ("Class year:", splitdata[2]) 
 

Output:

Student name: Tracy 
School: NYU 
Class year: Sophomore

Programming Challenge: The string below represents a series of test scores for a given student. Write a program that:

  • Prints out the average score for the student
  • Prints out the highest and lowest scores
  • Drops the lowest score and prints out the student’s average without that score

Click the "Run" button to see the program in action or download a copy.


Reading Data from the Web

You generally won’t be "hard coding" data into your programs as in the examples above. Instead, you will usually be processing information that comes to you in the form of user input or via an external data source.

One way to easily get information into Python from the "outside world" is to initiate a connection to the Web and obtain data via a URL (Universal Resource Locator, such as "http://www.google.com")

The general algorithm for reading data from a web-based source is as follows:

  1. Identify a source of data on the web that you wish to work with. Copy the URL that can be used to access that data source via a web browser.
  2. Create a new Python program that imports the urllib.request module.
  3. Initiate a web connection to an external data source using the urllib.request module.
  4. If the request is successful, obtain the data that exists at the URL. The data will be sent to your program in the form of single large string.
  5. "Clean up" the string into a meaningful form that your program can understand. This usually involves making sure that the character set used in the external file is one that your computer can handle (i.e. make sure that the data is being supplied as a set of ASCII characters).
  6. Parse your string into a list using the split method discussed above.
  7. Process this list and generate output.

Here is a sample bit of code that reads data from the Google website and displays it using Python.

import urllib.request

# where should we obtain data from?
url = "http://www.google.com"

# initiate request to URL
response = urllib.request.urlopen(url)

# read data from URL as a string, making sure
# that the string is formatted as a series of ASCII 
charactersdata = response.read().decode('utf-8')

# output!
print (data)

Keep in mind that the web is a volatile place – servers can go up and down at a moment's notice, and your connection to the network may not always be available. You need to be prepared to deal with connection errors in your programs. You can use the try / except / else suite to help “catch” errors before they crash your program. For example:

import urllib.request

# where should we obtain data from?
url = "http://www.google.com"

# attempt to access www.nyu.edu
try:

	# initiate request to URL
	response = urllib.request.urlopen(url)
    
	# read data from URL as a string, making sure
	# that the string is formatted as a series of ASCII 
	characters
	data = response.read().decode('utf-8')
    
	# output!
	print (data)
    
# an error occurred!  handle it here
except:
	print ("Something went wrong!")

Programming Challenge: Write a Python program that asks the user to type in a word in English. Determine if that name is one of the most 100 popular English words. Here is a link to a data file on the web which contains this information - the words in this file are sorted in order of popularity (i.e. the first item is the most popular word, the second is the second most popular, etc): http://cims.nyu.edu/~kapp/python/most_popular_words_in_english.txt

Here's how to get started:

  • Read in the data and print it out (use the sample code above to get started).
  • Next, split the data using the correct separator character (hint: there is a line break between each word).
  • Finally, use appropriate built-in functions and list methods to determine if the user's word is contained in the list that you generated in the previous step.

Click the "Run" button to see the program in action or download a copy.


Basic File I/O

Programs and Memory

All of the programs that you have been writing so far have reset themselves each time they run. This is because the data associated with the program (i.e. your variables) are stored in your computer’s memory (RAM). RAM is a volatile place – it serves as a computer’s short-term memory, and any RAM that is being used by a program is cleared when the program stops running. If you want your programs to retain data between executions then you need to find a "permanent" way to store and save information for future use.

A computer almost always has at least one type of long-term storage device at its disposal (usually some kind of hard drive). We can use the long-term storage capabilities of a computer to store data in the form of a file. Once we save a file it will remain on the long-term storage device after the program is finished running, and can be accessed and retrieved later on by the same program or by a different program. This is a very common technique that is used by almost all programs that need to keep track of some kind of information between executions. For example, when you click the "save" button in a word processing program you are storing the data you typed onto a long-term storage device.

There are two main families of files that we work with when writing programs - text files and binary files. Text files are files that contain data that is encoded as text (i.e. ASCII or Unicode characters) - data stored in a text file is visible and can be read by a program that is designed to view / edit textual data (i.e. a word processing program). We will be working with text files exclusively in this class.

Binary files contain data that is not encoded as a text format and thus are not "human readable". Binary files are intended to be read by other programs and not by humans directly and appear "garbled" when viewed via a word processing program.

Filenames and Working with File Objects

Every file on your hard drive must have a filename. On most operating systems a filename comes with a “file extension” which allows you to quickly tell what kind of data is stored inside that file. Extensions come after the last period in a file (i.e. “mydocument.doc” – “doc” is the file extension). File extensions tell the operating system and the user what kind of data is stored in the file and allows the OS to select an appropriate program to open a particular file. We will be working with text documents exclusively in this class, so all of our files will end with the "txt" file extension.

Now that we know how files are stored we can begin to work with them inside of Python. In order to access a file via your programs you must first created a special variable called a "file object" - this variable serves as a connection between your program and the operating system of your computer. File objects also store the name of a file that you wish to work with along with what you want to do to that file (write, read, append, etc).

Filenames are expressed as strings in Python. When you ask Python to open a file (i.e. “myfile.txt”) you are telling Python to open up a file with that name in the current directory. You can ask Python to open up a file that exists outside of the current directory by typing in an absolute path (see your book for more information on how to open files outside of your current working directory)

You can create a file Object by using the open function with following syntax:

# create a file object that opens the file "myfile.txt"
# for write access ("w")
file_object = open("myfile.txt", "w")

Writing Data to a File

You can write data into a file once you have created a file object and have opened a file for writing using the write function. Here’s an example:

# open a file for writing
file_object = open('myfile.txt', 'w')

# write two strings to the file
file_object.write("hello")
file_object.write("world")

# close the file when you’re done
file_object.close()

Note that the program above will create a text file named "myfile.txt" in your current directory which will contain the following data:

helloworld

You will probably want to try and avoid writing files that concatenate all of your data into one long line of unintelligible text. This is bad practice since it can be very difficult (or impossible) to extract out your data later on. One way to separate data in a text file is by splitting out your data into multiple lines using the \n escape character. This allows you to store a single value on each line of your text file, which will make things a lot easier for you later on when you need to read your file. Here's an example of how the \n character can be used to separate values onto new lines in a file:

# open a file for writing
file_object = open('myfile.txt', 'w')

# write two strings to the filewith linebreaks
# ending each string
file_object.write("hello\n")
file_object.write("world\n")

# close the file when you’re done
file_object.close()

Programming Challenge: Write a security program that prompts the user for a username and a password. Store the username and password into a file named "security.txt" - make sure to store the username and password on separate lines. Click here to download the solution.

Appending Data to a File

When you open a file for writing using the w flag you are asking Python to open the file so that you can place data into it. If the file does not exist Python will create it for you. If the file does exist Python will overwrite the current contents of the file. This isn't always what you want to happen! If you want to write data to a file but preserve the existing contents of a file you can open up the file in "append" mode. This mode allows you to "add on" additional data at the end of a file. Here's a sample program that shows this in action:

Critical: Make sure that if you open a file using the w flag, you are okay with the original data being erased! Many students have accidentally overwritten their homework assignments this way!
# open a file for appending
file_object = open("cartoon.txt", "a")

# ask the user for a cartoon character
character = input("Enter a cartoon character: ")

# write the character to the file
file_object.write(character)

# write a line break
file_object.write("\n")

# close the file
file_object.close()

Additional File I/O Techniques

Reading data from a file

You can read data contained inside a file once you have created a file object and have opened a file for reading using the read function. Here’s an example:

# open a file for reading
myvar = open("test.txt", "r")

# read in all data as one long string
alldata = myvar.read()

# output
print (alldata)

# close the file
myvar.close()

The read function extracts all data from a file as a string and returns a string to your program. This one large string generally needs to be further processed before it can actually be used. Often we can use the split method to cut apart the string into usable chunks.

Programming Challenge: Write a program that opens the "security.txt" file you created for the previous programming challenge and read in the username and password store in the file. Store these values into a series of variables

Next, prompt the user for a username and password using the input function. If the values supplied by the user match the values stored in the file, allow them to continue. Otherwise present an error message. Click here to download the solution.

Writing Numeric Data to a File

The write() function only accepts strings – the following code will generate a runtime error:

# open a file for writing
myvar = open("test2.txt", "w")

# write an integer to the file
myvar.write(55)

# close the file
myvar.close()

You need to convert any data that will be stored in a file into a string by using the str conversion function. Example:

# open a file for writing
myvar = open("test2.txt", "w")

# write an integer to the file in the form of a string
myvar.write(str(55))

# close the file
myvar.close()

Reading Numeric Data from a File

When you are reading data from a file you will often need to convert strings to a numeric data type if you want to perform calculations on the data. This process is the same as the process we have to go through when using the input function to read numeric data from the user.

Note: The int and float functions automatically remove line break characters for you!

Programming Challenge: Write a program that opens up a file named "testscores.txt". This file contains the following information in the following format:

student name
score1
score2
score3

Read in the values and print out the average score for the student specified in the file along with the student’s name. Click here to download the solution.


Quiz

Now that you've completed this module, please visit our NYU Classes site and take the corresponding quiz for this module. These quizzes are worth 5% of your total grade and are a great way to test your Python skills! You may also use the following scratch space to test out any code you want.

Feedback

Tell us what you thought about this module (it's anonymous).
How helpful did you find this module on a scale of 1-5:
Very unhappy face
Unhappy face
Neutral face
Happy face
Very happy face
Which resource(s) in this module did you find the most helpful (check all that apply):

Copyright 2014-2018