Tag Archives: RESTful

Experimenting with the AWS S3 API

Last month I uploaded a community sample that showed how to call an Amazon Web Services RESTful API, in particular for their S3 storage service.  That sample is contained within a single form, and is accompanied by some simple instructions and notes on assumptions made etc.  I used a form component type, and constructed it to use operations for the actual API calls, so that it would be easy to understand, and easy to modify to a service component, for wider usage.

The next thing I wanted to try was to provide the same functionality from a DSP.  Initially this could have meant replacing the Windows GUI layer with a HTML5 based layer in a DSP.  However, DSPs make the Uniface JavaScript API available, and thus there is an opportunity to try out the AWS SDK for JavaScript (in the Browser).  Information is available at http://docs.aws.amazon.com/AWSJavaScriptSDK/guide/browser-intro.html .

The main advantage of using this SDK is that becomes possible to avoid a lot of low level coding with the RESTful API.  If you study the form sample I mentioned earlier, you will see a lot of code to build canonical requests, and then to sign them securely.  This is all buried inside the various SDKs that AWS provides.  This was worth a try!

As it turned out, coding the JavaScript to list the bucket contents, download and upload files, was relatively easy.  In particular, the feature to generate signed URLs for downloading files is very handy.  In fact most of the buttons on the sample DSP have browser side JavaScript which calls the AWS SDK without much reference to the Uniface JavaScript API.  This just means that in some circumstances you might not need to use DSPs at all, but if your use case does involve exchanging information with back-end processes, then this sample should be of interest.  One such use-case is to save S3 files on the back-end server, and so a JavaScript activate is done to send the signed URL to a DSP operation, to complete the download.  In any case, it is tidy to keep the JavaScript code in the Uniface repository as much as possible.

So … although the JavaScript coding turned out easy enough, the challenge turned out to be how to authenticate the SDK calls.  In the form sample I used the AWS Access Key ID and a Secret Access Key to sign requests.  These were quarantined from the form source code, and the runtime user (who shouldn’t have access to the Uniface debugger), by storing the sensitive data in assignment file logicals.  Not the ultimate form of protection, but adequate for my sample.  The JavaScript SDK requires access to these artifacts, and since this runs in the browser, it exposes them to all users.  To slightly obscure these private values, I placed them in a separate JavaScript file, which is not referred to in the HTML, but dynamically loaded by Uniface with this statement:  $webinfo(“JAVASCRIPT”) = “../js/aws_config.js” .  Of course you can read the variable contents with any browser debugger.  So this DSP sample comes with a similar caveat to the AWS recommendations.

The options for supplying credentials to the AWS JavaScript API are described here:   http://docs.aws.amazon.com/AWSJavaScriptSDK/guide/browser-configuring.html .  So for my sample I did effectively supply hard-coded credentials for an IAM user that has read-only permissions.  Real applications will want a more secure method.  I was going to evaluate AWS Cognito, but it is not yet available in my region.  Another option to investigate is to use Temporary Security Credentials, via the AWS Security Token Service.  Further discussion on authenticating credentials is beyond the scope of this blog / sample.

One final security configuration had to be made, because the sample is running within a browser, which is likely to be enforcing CORS.  This is best explained in the documentation at http://docs.aws.amazon.com/AWSJavaScriptSDK/guide/browser-configuring.html#Cross-Origin_Resource_Sharing__CORS_ .

To summarise, Uniface developers have a choice when integrating with AWS.  They can choose the RESTful APIs for lower level control, in a wider set of situations, or they can use the JavaScript SDK for easier integration when using the Uniface JavaScript API.

Using CouchDB with Uniface

I won’t repeat any definitions of what NoSQL databases are, nor a review of any specific products. I’ve read plenty about NoSQL databases and I think that the general view of developers is that it is one more tool in the arsenal of application development. I generally believe that you should choose the right tool for the job at hand.

So, you may get that task one day where the advantages of using a NoSQL database outweigh the disadvantages. Can, or how can you use this database with Uniface? The answer definitely depends on the specific NoSQL database product. Between them all, they have a large variation in their APIs and data structures. For that reason I will just describe my experiences doing some prototyping with CouchDB from Apache. Be aware that this is slightly different to Couchbase, which appears to be a commercialised offshoot from what Apache took on board as an open source project. For brevity, I refer you to the website for information about CouchDB’s characteristics:   https://couchdb.apache.org/

The major characteristic of CouchDB is that the documents stored in the database are in JSON format. While investigating another project, I stumbled upon a convenient source of JSON formatted documents that I could use to store in my CouchDB database. I hope that you aren’t offended by simple Chuck Norris jokes. It is a unique genre that not all will enjoy, but it served my purposes adequately. Thus in studying my prototype, you could imagine how you would handle more business related data.

I have provided a sample form in the community samples part of Uniface.info. All you need to do, besides compiling that one form, is to download and install CouchDB from the earlier link provided. I downloaded the Windows version. I manually created the “cnjokes” database using the CouchDB provided Futon utility, also installed with the Couch DB. I also manually defined the design document “vcnjokes”; more about that later.

The top part of the COUCHTEST01 form is really a “utility” area, where you can manually enter URIs and run requests against the “cnjokes” database.   These requests use the UHTTP internal Uniface component. The way the CouchDB API is structured gives you a very RESTful web service interface, though there are some comments on how RESTful CouchDB really is, within their online tutorials. The results of the calls are available in the message frame.   You can press the GET button without adding anything to the URI and you will see some global information about the “cnjokes” database. Overall, this “utility” is not as flexible as the CouchDB provided Futon utility, but it might be helpful during further Uniface development.

The 4 buttons, and accompanying entity and fields provide the real prototype; effectively demonstrating a CRUD lifecycle of managing CouchDB documents. The UHTTP component is used to obtain a CN joke in JSON format from an internet website, and then the UHTTP component is used to interact with the localhost CouchDB server. The document format is deliberately unchanged between the external joke website and CouchDB. However, you could manipulate the JSON before storing it in CouchDB if required, using Structs. Note that I have used $uuid as the basis for assigning a document ID.

The other 3 buttons query the CouchDB database using views. Ad-hoc queries are not possible in CouchDB. The 3 views are defined in a single “design document” called “vcnjokes”. The source for that design document are provided with the sample download, as comments for the COUCHTEST01 form.

  • Button “Get all current jokes from CouchDB” uses view “joke_by_jokeid”. All jokes are retrieved, and sorted by joke_ID, but only a few columns are selected. It cannot be edited as the revision ID is not available. Note that escaped quotation symbols in the data are displayed as quotations.
  • Button “Get all nerdy jokes from CouchDB” uses view “nerdy_joke”. The jokes list is filtered to those that have a category of “nerdy”. This list also cannot be edited.
  • Button “Get all current data from CouchDB for edit” uses view “all”. This view references all of the document and so all fields, including revision ID, are available. Thus editing can be done, and when stored, the new revision number is updated. Note that escaped quotation symbols appear as stored, for ease of updating.

When preparing the JSON for display in a Uniface form, it is certainly necessary to use Structs to manipulate it into the Component data structure. In fact the choice of the external data structure of the form entities is quite arbitrary. CouchDB has no fixed schema. Thus you can never be sure that an external application won’t add data that renders your entities and fields obsolete. All I could do is generate a useful number of jokes, and observe that some of them have one category with a value of “nerdy”. However, I can see that the category is defined as a JSON array, and so I make sure to set   $tags->jsonClass = “array” before converting the struct to a JSON string. This is what led to the one to many relationship between CNJOKE and JCATEGORY. With my CouchDB data set, I verified my schema by manually adding several extra categories to some jokes, using structure editor commands to add and remove occurrences (tool tip text will assist you).

Hopefully this prototype demonstrates how modern features in Uniface allow integration with other modern software systems.