API Design

          We are now diving into the most interesting part of the class- creating the iHale software. To start, we needed to design both the REST and Java API for the software. An Application Programming Interface or API according to wikipedia, "is a particular set of rules and specifications that a software program can follow to access and make use of the services and resources provided by another particular software program that implements that API. It serves as an interface between different software programs and facilitates their interaction, similar to the way the user interface facilitates interaction between humans and computers".
         
          The whole process was a learning experience. I never really like reading APIs specially the really long ones that I usually get lost to. The reason for this could either because I don't like reading at all or the API was poorly designed. If the latter is the reason, maybe because it was really hard to create one. Well, it was hard when starting from scratch specially for me and the rest of the class since we never created one before. What made it even harder is because we are lacking other information. The information on the hardware side (sensors etc.).  Just like the definition said API is an interface for the interaction between programs. It is hard to create an interface if you don't know the capability of the program on the other side (like what data type is used). We assume that all data are represented in XML format, which is in our advantage. We've been working on parsing and creating XML since the beginning of the class.
         
          The most interesting part on creating API for me is just the reason why we are creating API. It is because we're creating a software. I never thought that I would ever come to this point that we are building a software with real purpose.

Here's our sample API.

Berkeley DB Katas

After learning the REST for the last couple of weeks, we are now tackling the database persistency with the use of an open source called Berkeley DB. The last couple of Kata's that we did, data was temporarily stored in the program and then deleted after the program finished running. This week, we are learning database persistency, which stores the data permanently somewhere in the program just like a real database. Below are the Katas that we needed to do to get familiar with persistency.

TO DO: Kata 1: Timestamp as secondary index

The example system provides a single key for retrieval of a Contact: it's unique ID. For this Kata, we will first implement a secondary key for Contacts that consists of a timestamp represented as a long value. For example, (new Date()).getValue() returns the current time as a long value.  This timestamp could mean a variety of things in practice, such as the time the contact was created, or the contact's birthday, etc., but we won't worry about that for now. 

Write unit tests to make sure that your secondary index works correctly, and that in particular you can retrieve a set of Contacts whose timestamps fall within a specified range.  

Once implemented, extend the ContactClient system with two new commands:

• get-timestamp: returns the Contact with the given timestamp, if such a Contact exists.
• get-range: returns the set of Contacts with timestamps between two specified values. 

Note that all of the sensor data in the Solar Decathlon system will have timestamps associated with them, and a frequent operation on the database will be to retrieve the sensor data instances during a given time period, so this kata will help you understand how to do that.

RESULT: This Kata was very confusing at first specifically the relation that we need to use for the secondary index. We decided to chose ONE_TO_ONE. It took us about half an hour to decide on this. We first tried to use ONE_TO_MANY but it needs an array or collection to hold the key. This made the addition of secondary key a lot harder. Since I don't any experience with databases,  I over think this simple problem and cost me to consume more time finishing this Kata. It took me more than 2 hours to finish this Kata.

TO DO: Kata 2: Wicket, REST, and BerkeleyDB

Create a client-server system for managing contacts that:

1. Provides a Wicket client.  The wicket client should consist of a single page, and allow the user to store a contact given its info, and retrieve a contact given its unique ID. The Wicket client code should invoke an underlying Restlet-based module to send a request to a ContactDB server to put or get data.
2. Provides a Restlet and BerkeleyDB server.  The server should accept and process requests using Restlet, and use BerkeleyDB to persist the Contacts.

Below is the diagram on how we suppose to accomplish this task.

RESULT: This Kata as expected is the hardest. This pretty much the combination of what we learned so far from the last semester to this current one. This is also the introduction on what we're going to be working on for the rest of the semester- the Home Management System for the Team Hawaii's house. Each system that we learned so far will be part of the web-based services:  user interface (Wicket), communication (Restlet), and persistent storage (BerkeleyDB). The way I looked at this Kata is in a more familiar way: Wicket is HTML, Restlet is PHP and Berkeley DB is SQL. The idea is similar where HTML display the UI and uses PHP to communicate to the SQL database, but the implementation are way different and a requires a lot more work. There are a lot of stuff going on on this Kata. I adapted the Wicket web interface from the REST I Kata, making sure that all necessary API's and programs were downloaded. We were also warned that we needed to use session for this Kata which made it a lot harder because we have to read and research to make this work. This Kata cost me about a day (8 hrs) and lots of research and head bumping on the table.

Download: Berkeley DB Katas

REST II

After struggling with the first set of REST Katas we have another sets to be completed. This time is about contacts also known as phone book. We are to manipulate add/edit code to accomplish the following Katas:

TO DO: Kata 5: Contacts resource

Modify the system to support a new resource called "contacts" and its associated URL: http://<host>/contactservice/contacts.  This resource only supports the GET operation and when it is invoked, it returns an XML representation of all of the contacts in the system.  To support the RESTful style of hypertextual links, your XML should look like the following:

<contacts>

  <contact>http://host/contactservice/contact/ID</contact>

  <contact>http://host/contactservice/contact/ID</contact>

</contacts>

Of course, we will substitute the actual <host> and actual <ID> for all <contact> elements in the returned XML.

Our system needs to programmatically determine the URL of the current host.  One way to do this is with code like the following inside your ServerResource method:

• new Reference(getReference(), "..").getTargetRef().toString()

We will also need to extend the createInboundRoot() method to support this new resource, and add a new package, class, and associated unit test.

RESULT: This is the hardest of all this part's Katas. I'm so confuse on how to do this. I didn't really know how to start implementing this task. With hours of research and help from other classmates I was able to accomplish this. The idea was to iterate through each contact and add it into the XML file. It took me about 4 to finish this Kata.

TO DO: Kata 6: Command line client manipulation of Contacts resource

Now that we have a contacts resource, extend the command line client to support a "get-all" and "delete-all" operation.  

The getall operation should invoke the GET contacts command to get the XML representation containing links to all the contacts, then extract the URL of each contact and perform a GET to retrieve its representation. Then print out the contact info associated with that URL (I.e. first name, last name, info). 

The delete-all operation should use the GET contacts command to get the XML representation containing links to all of the contacts, then extract the URLs, then call DELETE to delete them. 

RESULT: This wasn't bad to implement but it still took time to finish. Most of the time was used to research and try out some code that would work. The idea is to iterate to all the contacts in the XML and do either delete or get. There were a lot of resources that I found to do this but I had to tweak it to accomplish what I needed. It took me about 2 hours for this Kata.

TO DO: Kata 7: Add a telephone number to the Contact resource

Finally, extend the Contact resource to include a telephone number.  The telephone number must contain only numbers and the "-" character.  When clients do a PUT of a Contact representation in XML format, the server must check that the telephone number field has the correct format and set the Status field to indicate an error if the telephone number is incorrectly formatted. Write unit tests to check both valid and invalid PUTs of your enhanced Contact resource. 

RESULT: This is the easiest Kata. It was just a matter of adding all instance of phone number on all the modules. Although the easiest, it still took time to find the right place in the code to put the phone number. It me about 1 hour for this Kata.

DOWNLOAD: Restlet Katas II

REST

Classes just started which means a new semester of learning is coming. It didn't take that long before we actually dive deep into this learning experience. The first week of my class in Software Engineering focuses on REST and no, not that kind of rest, but Representation State Transfer. Wikipedia defines it as a style of software architecture for distributed hypermedia systems such as the World Wide Web. This is the first time that I've seen and heard of it and I'm sure most of the class too. We are learning REST to incorporate the project that we did last semester which is the front-end for the Team Hawaii's house's Home Management System. To be able to get familiarize with REST, we were ask to do some Katas.  Basically we have a DateClient and DateServer that we need to add/modify to be able to accomplish these Katas. Below are the 4 different Katas that we need to do.

TO DO : Kata 1: Time resources. Add three new resources called "hour", "minute" and "second" that return the current hour, minute and second.  

• Write unit tests for each of these resources and make sure they pass, and make sure you can run them both from within Eclipse and using Ant.  
• Make sure the DateClient system accepts these parameters. 
• Ensure that "ant -f verify.build.xml" passes with no errors.  

 RESULT: This is fairly easy. There's already a Day resources that exist that we need to copy and change part of it to retrieve/show hour, minute and second. It took me about 15 mins to finish this.

TO DO: Kata 2: Logging. By default, Restlet logs data about each request to the command line using the Java Logging API.  Read a tutorial (and/or google) to get familiar with the API, then do the following:

• define a ~/.dateservice/logging.properties file that enables the user to specify (among other things) that the logging information should go to a file, not the command line.
• tell the restlet-dateservice application to read and use the logging properties file in  ~/.dateservice/logging.properties

RESULT: This is a lot harder than the first one. Researching and reading tutorials took most of the time to accomplish this Kata. Some tutorial also varies and sometimes don't work on the current version or task that we needed to do. I have to create a property file inside a hidden folder in our home directory. I also needed to add a line inside the main method on the server to direct all logs into the same folder of the property file that I created. It took me about an hour to read tutorials and 30 minutes to actually code it. 

 

TO DO: Kata 3: Authentication. The current restlet-dateservice system performs no authentication.  Check the user guide (and/or google) for guidance, then implement HTTP Basic authentication for all the resources in the restlet-dateservice system.  To simplify things, all resources should be available as long as the requesting system uses HTTP Basic authentication and passes the username "guest" and password "pw". (You can hardcode that password into both the client and server sides of the system if you like.)

 

RESULT: Of course this is harder than the previous ones. I was able to find some tutorials online to help me accomplished this but none of them really are straight forward. I've tried different codes but none of them worked. I found this link that explained and showed the basic authentication in REST. It took me about 2 hours of searching and trying out tutorials and another 30 minutes to finish it once I found the tutorial that worked.

 

TO DO: Kata 4: Wicket. For this Kata, create a new client for the DateService server that is a web application using Wicket rather than a command line application.  The web application should bring up a single page with a single form that requests what aspect of the date (year, month, day, hour, minute second) the user desires. When the user submits the form, the web application uses Restlet to obtain the appropriate information from the DateService server, then presents the results to the user in the page.  

 

RESULT: To be honest, this Kata is the most I'm worried about. I wasn't able to figure this out. I tried couple solution like importing wicket in the build.xml file but no success. I'm currently still working on it, so the current distribution file for my program is incomplete. It took me about 3-4 hrs so far for this Kata.

Download: Restlet Katas

Solar Decathlon Group Design (Wicket)

Introduction

          This is the final assignment and group project for the semester. Our task is to create an actual web application of our Balsamiq Mockup that we did earlier in the semester (Mockup post can be found here). The idea of the project is to create a new version of the mockup using HTML, Blueprint CSS and Wicket. The task asked for a more real look and feel of the Home Management System for Team Hawaii's house entry in the 2011 Solar Decathlon.

The Project

          The project itself was like a roller coaster ride. Since we decided to choose my original design, I volunteered myself to take the challenge of creating the Base Page for the whole application and follow the original design. This gave an opportunity to hone my skills in designing and creating graphics and icons using Photoshop and I had fun doing it. Even though we have been learning Wicket for the past couple of weeks, it is still the most frustrating part of creating the web application in my own opinion. But I also found one advantage of using framework such as Wicket beside from learning JAVA. It allows us to create web applications without the need of an actual web server. It is also much easier to share the whole project trough a jar file or creating distributions. 

          The design of our web application changes as we slowly finding out some holes in the original design. One example is the navigation icons, which is originally placed vertically in the left side of the page. First, the placement is not very user friendly. The user have to browse all the way down the page to click the other icons and get to the other page. Second, this created a very restricted space for the content of the pages. If we have a small content in a page, there will be a big empty space below it. We decided to change this and moved the navigation icon on top of the page. This allows the user to see all the icons and made the pages re-size dynamically according to the amount/size of the content (See pictures below).  Our home page design also changed from being very very simple as our professor pointed out some design flaws and also after getting the feedback from Team Hawaii. We decided to create widgets of each element in the system. This allows user to quickly check the status of each entry without going through all the pages (See pictures below).

          We also ran into cross-platform and cross-browser problem. I'm doing the project on my Mac using Firefox and for some reason it did not display properly when my other group mates viewed it in their PC with Firefox. The fix to this is to put a note in the User Guide of our project saying that the web application is best viewed in PC using Firefox and in Mac using Safari.

Original Design

New Design

The Three Prime Directives of Open Source Software Engineering

          In creating the design of our project, we also need to consider whether we met the three prime directives of Open Source Software Engineering. Below are the three directives.

1. The system successfully accomplishes a useful task. - The system does not have to include every bell and whistle to accomplish a useful task. Indeed, the art of incremental development is to determine the smallest useful increment of functionality and implement that first.

          I believed that our project accomplished this directives as we created a web application that looks and feel like a fully functioning application. But what the user would say will matter the most.

2. An external user can successfully install and use the system. - The system must include sufficient user-level documentation to support download, installation, and use of the system without significant interaction with a system developer.

          We provided a documentation in out project site for user to be able to test and try out our project. I believed that it is sufficient enough to be able to say that we accomplished this directives, but then again users feedback will only verify this.

3. An external developer can successfully understand and enhance the system. - The system must include developer-level documentation providing insights into the design and development of the system that enable an external developer to understand and enhance it.

          Like the above directives, we provided documentation for the developers who wished to collaborate or check out our project. The code specially the Java files are well documented. While I believe that the HTML files could use some more cleaning up due to the fact that none of us in the group knows how to do HTML the right way.

Conclusion

           All in all, the whole project gave me more understanding of what Software Engineering is about. It all starts from the user/customer, in our case the Team Hawaii. Then we moved on to designing and creating mockup for the user to see and get feedback from. Finally the implementation which is the hardest yet the most exiting phase of Software Engineering. The whole project itself made me learned different applications, languages and technologies such as Photoshop, SVN, Project hosting, Wicket, CSS and HTML. I believe that there are a lot more room for improvement specifically on our design. We may need a lot more research and of course feedback from the user.
          The things that I wish I could have done differently is creating buttons instead of just hyperlink text in some of the pages (Security, Hvac). Also use Javascripts in some of the pages to add functionality specially in the buttons.

Download

Here's the site of our project where you can download our web application. Don't forget to read the User Guide if you want to test it out and Developers Guide to contribute, correct or even critique our project.

http://code.google.com/p/solar-decathlon-teamhawaii-4/

Wicket Katas

Introduction

This week's class' work is modifying Katas. Katas or specifically Code Katas are programming exercises that aims to refine ones programming skill through practice and repitition (Wikipedia).  Our class will be working on the previous exercises hoping to hone our skills in Wicket. There are 5 different Wicket Katas exercises that we need to work on and each exercise has about 1 to 3 needed modifications with the total of 11 modifications for the whole assignment. We needed to record the time that it took us to do each of these modification so let's go ahead and see how I did.

Example 01

TO DO: Kata 1A:  Add a new line to the page that says, "In one week, the time will be <time>", where <time> is replaced by a timestamp one week later than the timestamp that now appears on the page.  

RESULT: This took longer that I thought it would. There were some confusions on which method to use to get the timestamp for the next week.The original code uses Date() to retrieve the current date but the method getDate() is now deprecated so I had to do some reserching t able to do this Kata. I found the Calendar() and getCal() to do the job. This exercise took 20 mins.

TO DO: Kata 1B: Add a button to the page labelled "Refresh".  After pushing the button, the times update themselves. 

RESULT: This exercise is really easy. Just add a new button and set the onSubmit() to go the same page. This exercise took 15 mins.

TO DO: Kata 1C:  Wicket, by default, runs in "Development" mode, but production systems should run in "Deployment" mode.  Override the getConfigurationType() method so that Example01 now runs in Deployment mode.  See this page for details. 

RESULT: I was surprised with this one as I thought that this will be the hardest one. Luckily we were provided a link on the example of how to do this. This is accomplished by adding the getConfigurationType() method and setting the Application to Deployment mode. This exercise took 15 mins.

Example 02

TO DO: Kata 2A:  Add an additional link on the home page that says, "Go to image page".   Create this page, which should display an embedded image.  This image should be G-rated.  It should be in a .jpg file stored with the system, not retrieved from the web.

RESULT: All that is needed to do this is to copy the previous code and replace the label to create another link. Creating the page wasn't that bad either and for the image, It didn't took that long as I had to do this in the previous week's exercise. This exercise took 20 mins.

TO DO: Kata 2B:  Add a button on the home page with the label, "Make font bold".  After the user pushes it, all the text on the page should become bold, and the button label should change to "Make font italic".  When the user pushes that button, all of the text should change to italic and the button label should change to "Make font normal".  Pushing that button changes the text back to its original state and the button label should now say "Make font bold".

RESULT: This is the HARDEST of all the exercises. I worked on this for about 30 mins and decided to skip on this exercise. I had a couple of ideas on how to this. One of them is to create two additional pages. One page has all fonts in italic and another with all fonts in bold and links each buttons to the corresponding page. I'm not sure if this is even allowed or if this is a proper solution for this so I decided to do the very hard way. I created function that uses a switch statement to select the button's name. For the font type, I created if statements in the onSubmit() method. This will change the opening and closing font tag depending on what button was press. I had to ask for a lot of help on this one. The result is the initial button name is empty but when press, the button appropriate names will show up.This exercise took 4 hrs in span of three days. 

Example 03

TO DO: Kata 3A: Add a new tab called "Image" that takes the user to a page containing an embedded image (your choice, G-rated).  It should be in a .jpg file stored with the system, not retrieved from the web.

RESULT: This is pretty simillar to KATA 2A so this is one of the easiest. The only difference in here is that the page extends the BasePage, which has all the tabs. This exercise took 10 mins.

Example 04

Record how long it took you to accomplish each of these katas.
TO DO: Kata 4A:  Add a new cheese called "Velveeta", which costs $0.25/lb. 

RESULT: We chose this exercise to work on in class so I had help in finishing this. We base the code for this on the already existing entries. This exercise took 10 mins.

TO DO: Kata 4B:  Add a "country" field to the billing address that appears when checking out.   The country field should provide a drop-down menu with a selection of 5 countries.

RESULT: The same with the previous exercise, I had help to do this. The coding part was the easy part, but looking for the place on where to put it was the harder part. We had to configure 4 different files to accomplish this. This exercise took 15 mins.

Example 06

TO DO: Kata 6A:  Get rid of the blue columns that appear when displaying the website. These are for development, not deployment purposes.

RESULT: This is probably the easiest exercise. I had to first figure out where the blue columns are being created then where it's being called and then remove it from there. This exercise took 5 mins.

TO DO: Kata 6B:  Place the image underneath the form, not to the right.

RESULT: This is also an easy exercise. At first I tried to add a line break right after the form, but that didn't do anything. I played with some more and found out the span tag and replaced 16 to 20. This exercise took 5 mins.

TO DO: Kata 6C:  It is often convenient for web applications to consult a properties file when starting up in order to get configuration values.  An easy way to do this is with the standard Java Properties mechanism. For this Kata, modify your Example06 system to read in a file (if present) located in ~/.example06/configuration.properties.   (Note that ~ means "the user's home directory", and that there is a System property in Java that provides this value.)  This property file should contain a line like the following: deployment = true
In other words, it contains a property called "deployment" whose value is true if the Wicket application should run in deployment mode, otherwise the application should run in development mode (see Kata 1C). 
Your application should read in the properties file and set the System property wicket.configuration before starting up Wicket. The Jetty class is a convenient place to do this processing.
Finally, your application should write out a message on startup indicating whether or not it found the file, and what the resulting configuration mode will be.  If the file is not found, or the property is not present in the file, then the application should run in development mode.

RESULT: This is one of the hardest exercise in the whole assignment. This similar to Kata 1C, but this needed a lot more work to accomplish. This includes looking and reading from a file, catching exceptions and setting the Application properties. I was lost on how to accomplish this so I had to ask for some help. This exercise took 1 hr and lots of questions.

All in all, most of these exercises are pretty fun and easy to do and some are very frustrating that makes me wonder if it's worth a couple of hours to do. Luckily there are plenty of help so that made it ok. These exercises got me a little more comfortable on working on Wicket.

Downloads

Below are the finished distribution for all the exercises.

Kata Exercise 01

Kata Exercise 02

Kata Exercise 03

Kata Exercise 04

Kata Exercise 06

Wicket Chart (Google-O-meter)

Introduction

In addition to our group solar decathlon design mockups, this week, we also have to create a web page application individually. We are to implement a one page web application that provides two components: a form that accepts a variety of parameters along with a submit button that when pressed, generates a GoogleOMeter visualization.  Click here for an example of GoogleOMeter.

Web Application

The form should allow the user to set the start and end of the Y axis, the title, the width and height of the chart, and  the data value for the pointer. Our web application should use Wicket, Blueprint CSS for layout, the class' build system. We also need provide at least one useful test case and employ sessions so that multiple users can generate charts independently.

Wicket Experience

I was excited about learning Wicket. I just started  creating web pages and web application from my internship and I thought that Wicket will be very helpful in the future. I had to relearn HTML and then learn PHP, CSS and Javascript on my own to be able to do our previous project.  It turned out though that Wicket was harder than I thought. Setting the pages alone took me more than an hour. Since I never seen any Wicket code in action before,  I had to reuse the example Wicket project that were given to us by our professor and then work from there.
I have to say that creating web application/pages using Wicket is a very long and complicated process. The only thing I like about Wicket is we were able to work and enhanced our skill in Java. The only advantage that I can see about using Wicket is a high quality web application due to the fact that we are using build system to create it.
My over all experience in Wicket is not bad, because I get to learn a new framework and also enhance my skill in writing Java.

Wicket Chart Web Application

Below is the finished product for the implementation of the web application for creating Google O Meter chart. Beside from spending most of the time getting use to Wicket, I also spend most of my time making my page attractive by creating images for the header and the footer. I had fun creating this page and there are a lot more room for improvements.

 
Lesson Learned

Creating web application is very hard and  frustrating at first, especially when using a framework for the first time such as Wicket. It is very rewarding though once you see the result and it makes you want to learn some more.

Download

My Wicket chart can be downloaded here.