Author Archives: ontheroa

RESTful client sample code

I am providing a direct link to a wiki page where we have been keeping a list of sample code for various languages. Currently we have sample code for PHP, C#, Ruby on Rails, and Python. This link is usually provided within various UW Web Services documentation.

https://wiki.cac.washington.edu/display/bws/Sample+Client+Code

If you have sample code that you would like to add to the page then please add it as a comment to this post. I will also add it to the wiki page.

SWS Enrollment Enhancements

Last Thursday, 7/21, we deployed some enhancements to the SWS Enrollment resource. The update includes new attributes to the Enrollment & Registration resources, an upgraded version of our Core framework, and a handful of bug fixes. The latest Web Service (URI) version number remains unchanged at V4.

Please see our SWS client docs for details.

https://wiki.cac.washington.edu/display/SWS/Enrollment+Resource+V4

This is yet another UserVoice submitted request that has been acted upon. Please continue to submit your requests and votes to https://ontheroa.uservoice.com.

A non-RESTful experiment

As part of an ongoing effort to modernize the look and feel of the CSE web, we have begun to re-implement many of our internal tools in a more “Web 2.0-y” manner. The front-end tool we’ve chosen is ExtJS, a javascript framework that lets you create beautiful AJAX forms that communicate with your datastore using webservices.

The link between ExtJS and your backend data is handled via ExtJS Stores, Readers, and Writers. Stores are configured with the URLs of your webservice. One of the more interesting configuration options for these stores is restful, a boolean which tells the proxy whether to behave in a RESTful manner or using its own POST based CRUD operations. The default for restful is false.

The “Times Away” tool is an internal utility that CSE uses to allow faculty and staff to enter planned absences and which permits the display of a weekly calendar showing people who will be away from the department. We have using RESTful webservices for internal data exchange for some time, but when I recently re-implemented the Times Away tool, and I chose to write the backend in the non-restful way in order to see what insight I might gain.

Mechanics

The default data exchange scheme for ExtJS is quite simple. All interactions with the backend use POST (though this can be reconfigured to GET). Each POST contains an action parameter which can be any of the four well-known create, read, update, or destroy operations. Other POST parameters can be supplied as needed, enabling us to do typical things, like read all records for a given person.

ExtJS supplies subclasses for Json and XML readers and writers, both reasonable formats that are easily handled when writing your back end. Data from a webservice can be bound to forms and grids, and can also be read directly from the store should you desire such access. You need to make a few concessions in the format of your data (more on this below) but none of them is unreasonable.

Advantages of the Plain CRUD Webservice

  • Data are more tightly coupled to a traditional database architecture. You’re free to convert this to a disadvantage if you like. But in my mind, this is an application fed from a rectangular database table, and thinking in CRUD terms still feels the most natural.
  • Querying data is more natural.Asking a RESTful webservice for a subset of a resource is often accomplished using a query string. This can seem awkward and a violation of the purity of the RESTful esthetic. Knowing that all times away for boren might be at:
    http://my.server.com/times_away/boren

    is well and good, but finding his vacation days for the month of April might look like:

    http://my.server.com/times_away/boren?type=vacation&month=4

    On the other hand, supplying parameters specifying the desired user, type, and month, along with an action=read, seems perfectly self-consitent and in line with the way programmers commonly think.

  • Side-steps the POST/PUT problem. It has always bothered me that PUT is not widely supported by our HTTP infrastructure, leading to the need to overload the POST operation to support it.

Disdvantages of the Plain CRUD Webservice

  • Resources are not really addressable. For this limited application, this is not really a disadvantage, but it’s not going to scale well either. Web caches and search engines will respond well to http://my.server.com/times_away/boren, but not to http://my.server.com/times_away?user=boren.
  • Resources not easily browsable. One of the best parts of developing a
    RESTful webservice (or developing against one) is simply being able to type its address into your browser and have a look at it. With the CRUD webservice you’ll need to either write yourself a simple form or use some other tool.
  • Data format must be customized. For most imaginable uses, your payload needs to be formatted so that error conditions are communicated to the client. For an ExtJS application, reading boren’s vacation might return:

    {
    success: true,
    rows: [...]
    }

    or


    {
    success: false,
    message: "User not found"
    }

    While this format can easily be re-used in other ExtJS applications, it is not
    as universally portable and re-usable as a purely RESTful payload would be.

  • Error handling is fragmented. For the error condition, there is a discontinuity between an HTTP 404 meaning the service itself could not be found, and a returned success=false along with an application specific error code or description explaining the problem.

Conclusions

For an application of this limited scope, the CRUD webservice approach is fine. It offers many of the same advantages of a RESTful webservice: it uses HTTP for transport, decouples read/write operations from your datastore, and handles data in an easily human-readable format. Writing this webservice to be RESTful would have been a tiny bit more work on the back-end but not enough to matter. And the advantages of scalability, use of widely-accepted standards for communicating success and error conditions, and the ease of universal re-use outway the small extra effort.

Although I started this project with a fairly open mind, I half expected to
prefer the CRUD/POST by the end of the project, believing that it would be simpler to implement and understand. Instead, I ended up reinforcing many of the advantages of REST in my own mind. The next webservice I write will definitely be RESTful.

Introducing the new and improved UW Web Services Registry – now UW Web Services Portal

I am proudly announcing another incredible cross-departmental coding collaboration that resulted in an upgrade to our UW Web Services Registry (now a portal).

Team members:

  • Chris Heiland (UW Marketing)
  • Kilian Frey (UW Marketing)
  • Andrew S. McHarg (UWIT Network Tools)
  • Nick Chen (UWIT Network Tools)
  • Tony Chang (UWIT Application Integration Services)

After a one week (5 day) sprint, meaning we found a place to work face to face for one week with little to no interruptions, we were able to make the following changes:

1. New Developers – We got two new developers ramped up to help develop and support future updates to the UW Web Services Portal.

2. Amazon Web Services EC2 Micro Instances The UW Web Services Portal is currently running on an EC2 micro instance saving the UW about $50 per month and supporting a critical piece of the UW Web Service infrastructure for only about $17 per month.

3. New portal and registry design We now only have to give out ONE url to tell people about web services. Big thanks again to UW Marketing for helping our initiative with much needed UX help.

4. AWS EC2 SnapRest automation A tool that provisions a new test or production instance directly from our running production instance in only 5 minutes by running one command line. This is key for disaster recovery and cost reduction related to server provisioning. Chris writes up a blog on this innovative work.

5. Auto-complete Service Search A search text box has been added with the capability to search and filter services using an auto-complete design.

6. Upcoming Services Now you can add services to the Registry that are not yet in production and tell everyone that its coming soon.

7. UserVoice Integration We have integrated UserVoice feedback data into our site by using their RESTful APIs to correlate suggestions/feedback to and from our Portal for specific services.

8. Twitter We integrated our Portal with twitter, using their RESTful APIs, so that we can talk about web services as a community using tweets. Just hit up twitter hashtag #uwweb.

9. Blog RSS We integrated our OnTheROA blog into our Registry via RSS feeds, yes you can consider it another RESTful API.

10. Proven cross-department collaboration We have proven yet again that people from diverse UW teams, across organizational boundaries, can work efficiently and collaboratively in quick order to provide real value to the UW. Oh by the way and have FUN doing so!

You can read in detail about all the hard work here: https://sig.washington.edu/itsigs/Registry/Portal_Sprint_Feb_2011

You can also get a list of all our GitHub code checkins here: https://github.com/tonychang/uw-registry-v1/commits/master

Backups on Amazon Web Services (AWS)

One of the tasks on the schedule for the Web Services Registry was to move from a small instance to the newly offered micro instance on EC2. The migration to an expandable drive was the largest amount of work, however, a huge benefit as an instance mounted with an EBS root volume enables any size drive up to 1TB.

This also changed how backups were done. Instead of creating an image.manifest and uploading the resulting data to S3, it’s now done via snapshots. However, we still retained the ability to create an working server based on a current instance. The web console allows most of this activity but the real power comes in the command line tools.

For single operations the web interface is extremely easy to use but when dealing with hundreds of servers there is nothing better than accessing the AWS API. For this exercise we have only one server, but for efficiencies sake we are automating the mundane of the work. The current process starts with running the script, creatively called snaprest.

Once finished we can login to the control panel and see the results. The script takes a snapshot of the current drive, and registers it as an AMI. If we just wanted to backup the entire server we would be done. However one of benefits of this process is creating a additional environments for any necessary testing.

The AMI can now be launch as an instance and accessed just like the server it was generated from. We can launch a development environment for deployment of new code for testing in very short order. There is an elastic IP standing by so we can access the site via the development url so the server acts exactly like production. Switching development to production is also feasible via a few simple clicks.

The entire process is quick and due to the command line tools, entirely scriptable. Deployment is remarkably fast and reliable, the command line tools give plenty of flexibility and the process is very straightforward. No wonder so many companies are using AWS as a cloud solution for server infrastructure. Welcome to the future.

SWS XML & JSON – Ready for Testing

A build of Student Web Service with XML/JSON/XHTML payloads for all resources (with the exception of GradeRoster) is ready for testing. This is in response to many of you who shared with us your thoughts via Uservoice, emails, chat, and/or face to face meetings. We understand that to help advance developer adoption of web services it is important we offer XML and JSON representations of our web services.

Things you should know:

  1. Production release scheduled for the end of this month.
  2. All uris, including public, have remained unchanged for Xhtml. Please see below for examples on how to get XML and JSON representations.
  3. Although we have a good build ready for testing there are still some bugs and design considerations we will likely fix before we release to production.
  4. If you already have access to the web service then no new authorizations are required from the Office of the Registrar.
  5. We plan on decrementing Xhtml some time in the future and will provide ample communication prior to that change.

SWS EVAL Test Environment examples:

1. Using Course Search to find all Math courses using JSON or XML. Append .json or .xml to the end of the search resource name.

JSON
https://ucswseval1.cac.washington.edu/student/v4/public/course.json?year=2011&quarter=winter&future_terms=0&curriculum_abbreviation=math&course_number=&course_title_starts=&course_title_contains=&sort_by=on&page_size=10
XML
https://ucswseval1.cac.washington.edu/student/v4/public/course.xml?year=2011&quarter=winter&future_terms=0&curriculum_abbreviation=math&course_number=&course_title_starts=&course_title_contains=&sort_by=on&page_size=10

2. Get public course information for Math 100. Append .json or .xml to end of the uri.

JSON
https://ucswseval1.cac.washington.edu/student/v4/public/course/2011,winter,math,100.json
XML
https://ucswseval1.cac.washington.edu/student/v4/public/course/2011,winter,math,100.xml

Developer documentation for these new formats is available here. The target date is about 2 weeks from now so please help us find bugs before ship date.

Your timely feedback and bug notifications will be greatly appreciated to us and the rest of the UW. Please send all issues and bugs to sws-support@u.washington.edu.

Continue reading

A mildly contrarian view of REST

Blogger William Vambenepe takes the stand that Amazon proves that REST doesn’t matter for Cloud APIs. He states that he doesn’t mean it’s a poor choice, but that other reasonable alternatives (e.g. RPC) are just as good. Regarding AWS, he asks:

Has this lack of REStfulness stopped anyone from using it? Has it limited the scale of systems deployed on AWS? Does it limit the flexibility of the Cloud offering and somehow force people to consume more resources than they need? Has it made the Amazon Cloud less secure? Has it restricted the scope of platforms and languages from which the API can be invoked? Does it require more experienced engineers than competing solutions?

This Thursday: Web Service Discussion Group Ignites!

What: Our first of three quarterly Ignite-style presentations from UW people using Web services –- 5 minutes and 20 slides each! Each presentation will be followed by a 5 minute Q&A period. We also have a slightly longer session scheduled on the Family Educational Rights and Privacy Act (FERPA), a topic of importance to anyone who deals with student data.

When: Thursday, December 2, 2010 from 3:30-5pm.

Where: Odegaard Undergraduate Library, Room 220

Who: Here’s the agenda…

  • Virjean Edwards – FERPA and You
  • Dan Boren – Web App Security: Two Things You Gotta Know
  • James Renfro – RESTful Workflow
  • Josh Ritter – Google Sites Integration via RESTful APIs
  • Paul Schurr – The Genesis of XML and JSON Web Services
  • Heather Sherman – Reporting with Google Visualization
  • Jody Tate – Web Service Views Using jQuery UI

———————————– Web services discussion group info ———————————–

We meet once per quarter, from 3:30-5:00 PM in Odegaard Undergraduate Library, room 220. The meeting dates are: 12.2.10, 2.17.11 and 5.19.11.

To discuss Web services of all flavors, please email questions, suggestions, challenges, etc. to: appdev@u.washington.edu

Our Web service blog is located at: http://depts.washington.edu/ontheroa/

The UW Web services Registry is located at: http://webservices.washington.edu/

And finally, don’t forget a free service available to the UW developer community–Web Service Design Reviews. If you are designing, developing, or integrating a Web service and would like some help, call on us!

More information is available here: https://sig.washington.edu/itsigs/Web_Services_Design_Review

–Tom Lewis