Monthly Archives: December 2013

Summer and Fall 2013 Recap

The below posts are a recap of my activities since May 2013, when I last updated this blog.
Posts are listed in the order in which the events occurred.

My Undergraduate Project So Far, Part 2: Fall 2013

This post documents the progress I have made on my undergraduate project over the last four months, with Dr. Philip Johnson of the Collaborative Software Development Laboratory as my advisor. It is a continuation of Part 1.

September 2013: WattDepot Backup Script

September was spent working on a shell script and cron job to back up the WattDepot database, and documenting the process for restoring it to a new WattDepot installation. The script (viewable here) dumped WattDepot’s local PostgreSQL database, created a monthly backup of it if it was the first day of the month, created a daily backup, and deleted any backups more than seven days old.

During September, I also worked on the first drafts of the usability survey that would accompany my usability tests. The study had been redesigned to center around Vagrant and another usability enhancement by the Collaborative Software Development Laboratory’s Carleton Moore, called the Smart Grid Game Designer (SGG Designer).

October 2013: Vagrant Maintenance Updates and Experiment Redesigns

In October I discovered that VirtualBox 4.3.* versions were incompatible with the Vagrant virtual machine that I had been using because they had been packaged with an older version of the VirtualBox Guest Additions (4.2.0). The fix I implemented required the user to install the third-party vbguest plugin, which automatically attempts to upgrade all Vagrant machines to the version of the Guest Additions currently installed on their system. After this was complete, the user had to run the Vagrant installation script once again after editing some configuration files.

Since this solution was more complicated in the short term than continuing to rely on the last compatible version of VirtualBox, documentation was added telling users not to use versions newer than 4.2.18. The documentation for Vagrant virtual machine configuration was also updated to reflect changes to the behavior of the vagrant up command since Vagrant 1.3.5.

In October, I also conducted informal “friends-and-family” testing of my study design. The original plan had been to have each subject complete the Vagrant installation process in one virtual machine, then configure a Makahiki competition using the Smart Grid Game Designer in a second machine. It took one user approximately two hours to get through the entire process, which led me to break the test into two separate tests, each one being taken by a different group of subjects. The first test would now cover the Vagrant setup process, and the second test would now cover the SGG Designer. Documentation for Vagrant was simplified and revised based on feedback that described which parts were too complicated.

Last of all, I implemented a partial fix for a Makahiki bug that allowed mixed-case usernames to be used even though their users could not use them to log in due to a Django bug. The patch changed the handling of bulk user uploads to automatically convert mixed-case usernames to lowercase.

October – November 2013: IRB Submission and Django 1.6 Upgrades

The second half of October saw a long back-and-forth dialogue with the university’s Institutional Review Board that eventually resulted in the exemption of my study from full review, even though the delays forced me to move the study to January. My UROP research funding was approved once the IRB approved my exempt status, but I am still waiting on the actual disbursement of funds.

In November, Yongwen Xu of the CSDL upgraded Makahiki to use Django 1.6. I uncovered and fixed some minor bugs related to Vagrant functionality, but otherwise my part of the project was mostly unaffected by the upgrade.

December 2013: User Interface Work Resumes

Towards the end of the semester, I was finally able to resume work on the new configuration interface I had originally planned to implement to improve Makahiki’s usability. Some basic features like user uploads and team and group editing have been implemented in the Makahiki widgets on this page, but round settings and challenge settings are still not included here.

The current state of the simplified configuration interface. Only a few settings widgets are currently implemented.

In the next semester I will be conducting my study, completing my final research report and presentation for the Spring Symposium, hopefully finishing work on the configuration interface, and preparing to graduate with my B.S. in Computer Science.

My Undergraduate Project So Far, Part 1: Spring and Summer 2013

Over the last year, I have been working on an undergraduate research project for Dr. Philip Johnson, head of the Collaborative Software Development Laboratory (CSDL). This is the first part of an end-of-year recap of my current progress. Next semester I hope to maintain a more regular update schedule.

January – May 2013: Research Proposal

In the Spring 2013 semester, I completed a research proposal for usability testing of new features that I planned to implement in the Makahiki software and presented my work at the 2013 Spring Proposal Conference. The goal of the research was to develop and test additional functionality for Makahiki that made it easier for system administrators and developers to install and configure. In addition to a successful research proposal, I was approved for funding from the University of Hawaii’s Undergraduate Research Opportunities Program (UROP).

The initial research proposal produced for my Honors 495 research writing course created a 28-page report, a 15-minute presentation, and a 48-inch-by-36-inch poster. PDF versions of these files can be viewed below.

Final Proposal
Spring Proposal Conference 2013 Oral Presentation
Spring Proposal Conference 2013 Poster
If the spacing appears to be off in some documents, this is probably a result of the Dropbox PDF viewer. If this occurs, download the document to view it correctly.

My poster for the 2013 Spring Proposal conference. Click to view a full-size image (3.56 MB).

June 2013: SGSEAM Conference Paper

My first task for the undergraduate project was to assist with the data analysis for a conference paper on the “Serious Game Stakeholder Experience Assessment Method” (SGSEAM) under development by Dr. Johnson’s Collaborative Software Development Laboratory. This paper was published in October 2013 as “SGSEAM: Assessing serious game frameworks from a stakeholder experience perspective” [citation].

One source of usability testing data for this paper was the surveys that my classmates and I had filled out in ICS 691 when we tested out the Makahiki administrative interface. Having been lucky enough to avoid some of the more extreme problems when I installed and set up Makahiki as a student, it was interesting to be able to take a higher-level look at the data in order to identify trends in what we had seen as usability problems.

July 2013: Script Development

An analysis of the problems that my classmates and I had reported when we installed local instances of Makahiki in Spring 2013 indicated that many of us had wanted the Makahiki installation process to be faster or more automated. Dr. Johnson suggested that this could be fixed by creating an automated script-based installation for Makahiki.

This shifted the focus away from the Makahiki serious game setup feature which was part of the initial proposal, but allowed me to work on a more feasible virtualization-based solution to problems with the initial setup of Makahiki.

Two solutions were eventually attempted: automated Python scripts for Red Hat Enterprise Linux / CentOS 6.4 systems and Ubuntu 12.04.* systems, and a Vagrant installation with a provisioning script. The initial plan was to support all of these methods.

The Ubuntu Python script implementation used a single master script to run other sub-scripts based on the command-line arguments that it was issued. The scripts installed Makahiki dependencies, initialized and updated the database, and cleaned up files downloaded during the installation. This turned out to be somewhat difficult to port over to CentOS due to the differences in the CentOS 6.4 base installation of Python (2.6.6) and the minimum version needed by Makahiki (2.7.3).

The solution that I eventually worked out was to create an additional script for CentOS that completed an altinstall of Python 2.7.3. With an altinstall, Makahiki would run as long as a Python 2.7.3 environment had been initialized in the shell that makahiki was running in. However, to initialize this environment, the user had to remember to run the command "scl enable python27 bash" manually in each new shell. In addition, sudo commands ignored the altinstall and ran as if they were in the original shell with Python 2.6.6.

During script development, I found that Ubuntu 12.04.* and CentOS 6.4 systems’ default repositories used the wrong version of libmemcached, so compiling and installing the correct version had to be built into the scripts as well.

Due to the version-specific commands and dependency issues, Dr. Johnson and the other CSDL members advised me to focus on the Vagrant installation, as the OS-specific scripts were becoming unwieldy and would likely be difficult to maintain in the future. The scripts were removed from the project, but I was able to add the CentOS installation process to the Makahiki local Unix installation documentation.

The Python scripts are no longer being maintained but can be viewed on GitHub.

The Vagrant solution involved a single Ubuntu 12.04 LTS virtual machine downloaded from Vagrant’s servers. This machine was provisioned using a shell script that installed the necessary packages. Unlike the Ubuntu and CentOS Python scripts, which had been intended for possible use on non-virtualized installations, the Vagrant script could also overwrite configuration files with Makahiki-specific ones: it would always be running on a virtual machine, and always be overwriting the exact same default files.

Installing Makahiki on a Vagrant virtual machine on Windows was simpler than the process of installing Makahiki directly onto a Windows machine, so my next task was to document the configuration of an appropriate development environment for a Windows host working with Makahiki on Vagrant.

August 2013: Vagrant and Eclipse

In August, I documented the process for using a Vagrant installation of Makahiki with an installation of Eclipse on the host machine. The resulting environment used the PyDev and Remote Systems Explorer plugins to enable Python and SSH connection support, and installed XML/HTML/Javascript editors to assist with the process of web development for Makahiki.

The remote debugger works well enough, but I wonder if it is really worth bringing down your firewall.

The PyDev remote debugger was blocked by Windows Firewall even when a rule was explicitly created to allow its designated port to pass through, and only worked when the firewall was fully deactivated. The remote debugger instructions were added to the documentation, but with a warning that using the debugger would likely create a security risk.

As the Fall semester began, I shifted my focus away from direct Vagrant development and worked on getting my study proposal approved by the university’s Institutional Review Board.

This summary of the progress of my undergraduate project continues in Part 2.

Greyhats: Fall 2013 Recap

The Greyhats club is a Registered Independent Organization at the University of Hawaii at Manoa. It was founded in 2010 by five students who have since graduated. I have been a member of this club since Spring 2011, and have previously served as its Treasurer (Spring 2012) and Vice President (Fall 2012, Spring 2013).

The purpose of the Greyhats club is to educate its members in cybersecurity-related topics and to train for the annual Collegiate Cyberdefense Competition that happens every March. In the past, the Greyhats participated in the virtual regional, and I was on the CCDC 2012 team which placed 3rd. I was also a member of the CCDC 2013 team, which unfortunately did not place.

Though I prepared some basic Unix command tutorials this semester, I have stepped back from preparing instructional content for the club as my responsibilities for my undergraduate project have increased and I approach my target graduation date next semester. It is in the best interests of the club to have more active leadership that will be enrolled at the university for at least the next two semesters, so I did not run for President or Vice President. I am currently the Treasurer, a position which is largely ceremonial because the club did not do fundraising or apply for grants.

At the beginning of the semester, I configured a Google Groups forum for use as a discussion board for the club. This enabled us to move some discussions out of the mailing list to a format more suitable for longer conversations.

As in Spring 2013, I continued to serve as de facto Secretary by announcing meeting times, preparing meeting minutes, and distributing them to the mailing list. I also took up the management of the club’s new cloud file storage service account, uploading meeting minutes, tutorials, and presentations. I organized and uploaded documents from past CCDC events in 2011, 2012, and 2013 so that other members could use them to train for the 2014 CCDC. We have run into several situations this year where only one or two of us had some document that a former member had sent around. Creating an archive of tutorials and competition materials is a way for the club to try to preserve its institutional knowledge in a more organized way as members graduate.

Po’oihe Cyber Security Exercise

In July 2013, I attended planning meetings and contributed injects (competition exercises) for the Po’oihe Cyber Range’s first annual Cyber Security Exercise as a member of the White Team. The actual Po’oihe exercise took place from August 2-4, 2013 at the University of Hawaii at Manoa. The Blue Team played the role of the IT department for a Hawaii bank.

Certificate of participation (click for large image).

Teams

The Cyber Security Exercise followed a structure of simulated cybersecurity exercises that divided competitors, support staff, and judges into five categories.

The Five Teams You Meet at a Cybersecurity Exercise

Blue Team
A cybersecurity competition has multiple Blue Teams, each one made up of people from a specific company, school, or other organization. Pooihe had a mix of businesses, schools, and Armed Forces teams. Each team has a Team Captain and Team Co-Captain who were responsible for communicating with the White Team.
Red Team
The Red Team is a team of cyber security professionals who work to disrupt the operation of the computers that the Blue Team is responsible for. Once the team gains unauthorized access to a system, they can add and delete users, deface web pages, bring down critical services, plant keyloggers and other listening tools, or whatever else is needed for the exercise. Once a Blue Team detects that the Red Team has gained access to their system, they are usually required to file an incident report with the White Team that describes the impact of the Red Team attack, the evidence for the attack, and any steps taken to mitigate the attack.
White Team
The White Team serves as the judges for the event, scoring the performance of each Blue Team and enforcing the rules of the competition. At the Po’oihe exercise, the White Team also designed the exercises, or “injects” for the event.
Black Team
The Black Team is in charge of setting up the competition environment (equipment, rooms, etc.) in advance of the competition.
Gold Team
The Gold Team acts out the roles of the management and staff of the fictional company in all communications with the Blue Teams.

Injects

Each inject consists of a task (such as creating a security policy, setting up a web server, configuring a firewall, etc.), a time deadline, and a point value. Each inject is scored according to a separate document that defines how it will be scored. Injects are delivered to each Blue Team as printouts or by email at specific times during the competition. Each Blue Team’s total score on all injects is used to determine the overall winner of the exercise.

Injects are judged by the White Team during or after the competition. Teams will lose points for an inject if their results are incomplete, incorrect, or otherwise not sufficient to meet the criteria for the exercise. The overall score of a team for all of its submitted injects determines its ranking.

Jordan K. Takayama

Professional Portfolio