Category Archives: Undergraduate Project

Spring 2014 Recap

In Spring 2014, my final semester at the University of Hawaiʻi at Mānoa, I completed my undergraduate project, worked as a teaching assistant for an introductory cybersecurity course, and completed my undergraduate degree.

1. ICS Security Assistant: ICS 425, Cybersecurity and Ethics I

As a teaching assistant to Professor Barbara Endicott-Popovsky of the University of Washington, I supported the teaching of the course by posting and grading assignments, responding to student concerns, and managing student peer evaluations and weekly discussions of current events in cybersecurity. I had taken this course one year ago in the Spring 2013 semester, which prepared me well to communicate the course material to students this year.

Meetings with the professor to plot the course of the lessons, appraise my performance, and address student concerns occurred once a week, usually by phone.

This was a distance learning course, so I did not have set office hours. All discussion and communication with students occurred through email or forums managed on Laulima, a collaboration system for the University of Hawaii that is built on the Sakai platform.

2. ICS Undergraduate Project, Part 3: Spring 2014

As in the previous semester, my responsibilities as a member of the Collaborative Software Development Laboratory were to maintain the documentation for the Ubuntu, CentOS, and Vagrant installation methods, and to maintain the Vagrant provisioning script and its supporting files.

Resolving Compatibility Issues for Makahiki Dependencies

The release of pip 1.5 on New Year’s Day 2014 temporarily broke the Vagrant provisioning script (the bash shell script mentioned in previous posts). Makahiki was, and as of August 2014 still is, dependent on several external, unverified Python packages, including the libraries services for Django, PIL, NewRelic, Sphinx, and Markdown.

After changing the provisioning script to use setuptools-0.8 for compatibility with pip 1.5.x, I ran through the Ubuntu virtual machine installation process several times to determine which pip-installed packages needed to be flagged with “allow-external” or –allow-unverified” to be installed. The pull request describing the proposed changes can be seen here.

The development team decided that it was simpler to remain on the last known compatible version, pip 1.4.1, so I reverted the changes in my code and changed the documentation to install that older version instead (see GitHub pull request 592). However, the modified pip commands were needed to pass the CSDL’s Travis CI testing, which built the software using the newest version of pip (see GitHub pull request 603).

In early January, the sequence of commands used to install the Python Software Collections Library (SCL) stopped installing Python 2.7.3, and stopped installing correctly on i386-architecture CentOS virtual machines. This was the result of the maintainer of the SCL switching to Python 2.7.5 and ending support for i386 architectures.

At the time, Makahiki on CentOS required Python 2.7.3. To enforce this version requirement, I revised the installation process to compile and install Python 2.7.3 from source code after installing development packages. This was based on my previous work on the now-deprecated Python installation scripts for Ubuntu and CentOS, which had also compiled and installed Python 2.7.3 from source code. These changes were applied to Makahiki in March 2014 (see GitHub issue 586).

Towards the end of January, I was also a Smart Grid Game administrator for the 2014 Kukui Cup, a sustainability and alternative energy educational game supported by the Makahiki and WattDepot software. As an administrator, I was responsible for evaluating and scoring submissions for the activities and questions which students completed and submitted through the Makahiki Smart Grid Game module.

In late February, we were able to remove the VirtualBox version warning from the documentation after tests with Virtualbox 4.3.8 demonstrated that the bugs caused by VirtualBox 4.3.0 had been fixed (see GitHub pull request 603).

Though some work was done on the alternate configuration user interface, I spent March through May working on completing the data analysis and undergraduate thesis writeup for my Honors Upper Division project.

Conclusion of the Honors Upper Division Project

My undergraduate research was presented at the 2014 Spring Symposium, held on May 8, 2014. The undergraduate thesis was submitted to the Computer Science department and the Honors Department. Links to full versions of each document are provided below:

The main insight gained from the usability testing was that the Vagrant installation method that I had developed represented a significant time savings over the manual installation method, cutting over an hour from the average setup and configuration time. This was accomplished by using a single, default Ubuntu 12.04 LTS virtual machine as the environment for Makahiki, which enabled the installation of software packages and replacement of configuration files to be automated in Vagrant. This reduced the need for OS-specific installation steps, since all users of a Vagrant installation would be installing Makahiki onto the same virtual machine. The usability testing also helped to indicate areas of the documentation which were difficult to understand or likely to cause user error, and these areas were cleaned up after the study had been finished.

I received my B.S. in Computer Science with Honors from the University of Hawaiʻi at Mānoa, graduating on May 17.

3. The George Washington University

After a successful application to The George Washington University’s Cybersecurity in Computer Science master’s degree program, I am preparing to begin my studies at the end of this month. I was fortunate to be awarded a CyberCorps scholarship, and I am looking forward to experiencing the unique opportunities for academic and professional development that are available at the university.

My Undergraduate Project So Far, Part 2: Fall 2013

This post documents the progress I have made on my undergraduate project over the last four months, with Dr. Philip Johnson of the Collaborative Software Development Laboratory as my advisor. It is a continuation of Part 1.

September 2013: WattDepot Backup Script

September was spent working on a shell script and cron job to back up the WattDepot database, and documenting the process for restoring it to a new WattDepot installation. The script (viewable here) dumped WattDepot’s local PostgreSQL database, created a monthly backup of it if it was the first day of the month, created a daily backup, and deleted any backups more than seven days old.

During September, I also worked on the first drafts of the usability survey that would accompany my usability tests. The study had been redesigned to center around Vagrant and another usability enhancement by the Collaborative Software Development Laboratory’s Carleton Moore, called the Smart Grid Game Designer (SGG Designer).

October 2013: Vagrant Maintenance Updates and Experiment Redesigns

In October I discovered that VirtualBox 4.3.* versions were incompatible with the Vagrant virtual machine that I had been using because they had been packaged with an older version of the VirtualBox Guest Additions (4.2.0). The fix I implemented required the user to install the third-party vbguest plugin, which automatically attempts to upgrade all Vagrant machines to the version of the Guest Additions currently installed on their system. After this was complete, the user had to run the Vagrant installation script once again after editing some configuration files.

Since this solution was more complicated in the short term than continuing to rely on the last compatible version of VirtualBox, documentation was added telling users not to use versions newer than 4.2.18. The documentation for Vagrant virtual machine configuration was also updated to reflect changes to the behavior of the vagrant up command since Vagrant 1.3.5.

In October, I also conducted informal “friends-and-family” testing of my study design. The original plan had been to have each subject complete the Vagrant installation process in one virtual machine, then configure a Makahiki competition using the Smart Grid Game Designer in a second machine. It took one user approximately two hours to get through the entire process, which led me to break the test into two separate tests, each one being taken by a different group of subjects. The first test would now cover the Vagrant setup process, and the second test would now cover the SGG Designer. Documentation for Vagrant was simplified and revised based on feedback that described which parts were too complicated.

Last of all, I implemented a partial fix for a Makahiki bug that allowed mixed-case usernames to be used even though their users could not use them to log in due to a Django bug. The patch changed the handling of bulk user uploads to automatically convert mixed-case usernames to lowercase.

October – November 2013: IRB Submission and Django 1.6 Upgrades

The second half of October saw a long back-and-forth dialogue with the university’s Institutional Review Board that eventually resulted in the exemption of my study from full review, even though the delays forced me to move the study to January. My UROP research funding was approved once the IRB approved my exempt status, but I am still waiting on the actual disbursement of funds.

In November, Yongwen Xu of the CSDL upgraded Makahiki to use Django 1.6. I uncovered and fixed some minor bugs related to Vagrant functionality, but otherwise my part of the project was mostly unaffected by the upgrade.

December 2013: User Interface Work Resumes

Towards the end of the semester, I was finally able to resume work on the new configuration interface I had originally planned to implement to improve Makahiki’s usability. Some basic features like user uploads and team and group editing have been implemented in the Makahiki widgets on this page, but round settings and challenge settings are still not included here.

Current state of the simplified configuration interface.

The current state of the simplified configuration interface. Only a few settings widgets are currently implemented.

In the next semester I will be conducting my study, completing my final research report and presentation for the Spring Symposium, hopefully finishing work on the configuration interface, and preparing to graduate with my B.S. in Computer Science.

My Undergraduate Project So Far, Part 1: Spring and Summer 2013

Over the last year, I have been working on an undergraduate research project for Dr. Philip Johnson, head of the Collaborative Software Development Laboratory (CSDL). This is the first part of an end-of-year recap of my current progress. Next semester I hope to maintain a more regular update schedule.

January – May 2013: Research Proposal

In the Spring 2013 semester, I completed a research proposal for usability testing of new features that I planned to implement in the Makahiki software and presented my work at the 2013 Spring Proposal Conference. The goal of the research was to develop and test additional functionality for Makahiki that made it easier for system administrators and developers to install and configure. In addition to a successful research proposal, I was approved for funding from the University of Hawaii’s Undergraduate Research Opportunities Program (UROP).

The initial research proposal produced for my Honors 495 research writing course created a 28-page report, a 15-minute presentation, and a 48-inch-by-36-inch poster. PDF versions of these files can be viewed below.

Spring proposal conference 2013 poster.

My poster for the 2013 Spring Proposal conference. Click to view a full-size image (3.56 MB).

June 2013: SGSEAM Conference Paper

My first task for the undergraduate project was to assist with the data analysis for a conference paper on the “Serious Game Stakeholder Experience Assessment Method” (SGSEAM) under development by Dr. Johnson’s Collaborative Software Development Laboratory. This paper was published in October 2013 as “SGSEAM: Assessing serious game frameworks from a stakeholder experience perspective” [citation].

One source of usability testing data for this paper was the surveys that my classmates and I had filled out in ICS 691 when we tested out the Makahiki administrative interface. Having been lucky enough to avoid some of the more extreme problems when I installed and set up Makahiki as a student, it was interesting to be able to take a higher-level look at the data in order to identify trends in what we had seen as usability problems.

July 2013: Script Development

An analysis of the problems that my classmates and I had reported when we installed local instances of Makahiki in Spring 2013 indicated that many of us had wanted the Makahiki installation process to be faster or more automated. Dr. Johnson suggested that this could be fixed by creating an automated script-based installation for Makahiki.

This shifted the focus away from the Makahiki serious game setup feature which was part of the initial proposal, but allowed me to work on a more feasible virtualization-based solution to problems with the initial setup of Makahiki.

Two solutions were eventually attempted: automated Python scripts for Red Hat Enterprise Linux / CentOS 6.4 systems and Ubuntu 12.04.* systems, and a Vagrant installation with a provisioning script. The initial plan was to support all of these methods.

The Ubuntu Python script implementation used a single master script to run other sub-scripts based on the command-line arguments that it was issued. The scripts installed Makahiki dependencies, initialized and updated the database, and cleaned up files downloaded during the installation. This turned out to be somewhat difficult to port over to CentOS due to the differences in the CentOS 6.4 base installation of Python (2.6.6) and the minimum version needed by Makahiki (2.7.3).

The solution that I eventually worked out was to create an additional script for CentOS that completed an altinstall of Python 2.7.3. With an altinstall, Makahiki would run as long as a Python 2.7.3 environment had been initialized in the shell that makahiki was running in. However, to initialize this environment, the user had to remember to run the command "scl enable python27 bash" manually in each new shell. In addition, sudo commands ignored the altinstall and ran as if they were in the original shell with Python 2.6.6.

During script development, I found that Ubuntu 12.04.* and CentOS 6.4 systems’ default repositories used the wrong version of libmemcached, so compiling and installing the correct version had to be built into the scripts as well.

Due to the version-specific commands and dependency issues, Dr. Johnson and the other CSDL members advised me to focus on the Vagrant installation, as the OS-specific scripts were becoming unwieldy and would likely be difficult to maintain in the future. The scripts were removed from the project, but I was able to add the CentOS installation process to the Makahiki local Unix installation documentation.

The Python scripts are no longer being maintained but can be viewed on GitHub.

The Vagrant solution involved a single Ubuntu 12.04 LTS virtual machine downloaded from Vagrant’s servers. This machine was provisioned using a shell script that installed the necessary packages. Unlike the Ubuntu and CentOS Python scripts, which had been intended for possible use on non-virtualized installations, the Vagrant script could also overwrite configuration files with Makahiki-specific ones: it would always be running on a virtual machine, and always be overwriting the exact same default files.

Installing Makahiki on a Vagrant virtual machine on Windows was simpler than the process of installing Makahiki directly onto a Windows machine, so my next task was to document the configuration of an appropriate development environment for a Windows host working with Makahiki on Vagrant.

August 2013: Vagrant and Eclipse

In August, I documented the process for using a Vagrant installation of Makahiki with an installation of Eclipse on the host machine. The resulting environment used the PyDev and Remote Systems Explorer plugins to enable Python and SSH connection support, and installed XML/HTML/Javascript editors to assist with the process of web development for Makahiki.

The PyDev remote debugging server.

The remote debugger works well enough, but I wonder if it is really worth bringing down your firewall.

The PyDev remote debugger was blocked by Windows Firewall even when a rule was explicitly created to allow its designated port to pass through, and only worked when the firewall was fully deactivated. The remote debugger instructions were added to the documentation, but with a warning that using the debugger would likely create a security risk.

As the Fall semester began, I shifted my focus away from direct Vagrant development and worked on getting my study proposal approved by the university’s Institutional Review Board.

This summary of the progress of my undergraduate project continues in Part 2.