Witty and HTTP Client

By approaching this week we can finally say that we have completed the first half of the internship successfully . According to our university internship program , the second half is dedicated for the student to carry out a specific project that is related to the host company, so here we are, in this week we had the opportunity to sit with Dr.Shawn discussing about the project and what should be accomplished next. The project I’ll be doing is related or we can say it is a complement of what I have been doing so far.

So for the next few months and for my SIP project I’ll be dealing with handling the verilog top level module from synthesizing it to getting it implemented in the Spartan-6 FPGA. And since this my first time to deal with the FPGA in general and Xilinx Spartan 6 specifically I’ll try to take you step by step and explain every single detail that I’ll learn during working in the project. I’ll dedicate my next blog entries for that but for now in this blog entry I need to get some other stuff done first :).

If you still remember, in one of my previous blog entries I have talked about the Emacs software and its verilog mode and how we can use emacs to elaborate the verilog top module. The Emacs can be run using the batch mode (no window opened) from the terminal which would help to use the Emacs AUTO feature to get the top level module elaborated from our Code using functions such as system() to run the shell commands. The command is simply :

emacs --batch -f fileName.v (Verilog file)  verilog-batch-auto

This command used to be run in the machine or the server that hosts the Web interface or the front-end application but from now on, all this work (elaborate the verilog top level module, synthesis it , compiling the user code) should be handled by another machine or server.So here we are again , back to web stuff , getting the two machine communicate with each other using the HTTP protocol.   The server code that handles the verilog file was previously written by Dr.Shawn. The code has a backend engine (WfileResource) that receive the HTTP request and based on the request and its path it will handle it. So my task was to adjust my frond-End code to be able to run as a client side and to send the (verilog file, C++ user code ,… etc) to other machine which will be dealing with it according to the HTTP request information.

HTTP Protocol and WITTY :

When it comes to HTTP protocol in Witty we can divide witty classes into two main categories:

• The Server side classes that deals with the server side aspects through WResource such as Request, Response , ResponseContinuation and UploadedFile classes .

• The client side classes that implements HTTP client and this category contains two main classes which are:

Client: an HTTP client

Message: a message to be sent with the client, or received from the client.

Previously I was mainly working with the server-side classes when we dealt with getting the code editor content to or from the server using the JQUERY library , I never explored how to form a HTTP request message because all these were done before by the Browser (The client) and I was always the interpreter , the receiver but never the direct sender or the request creator.
But now My front-End Engine should really be able to not only receive and send the request to the browser but also to be able to communicate with other machine Engine using the HTTP protocol.

HTTP Client

To implement an HTTP client in Witty as I mentioned above you really need to deal with witty classes such as Client and message. What I did was basically creating a function in my front-end editor Engine (Wresource inherited class ) that does the client job. The function will simply take the file name and the URL that it should sent to as parameters. Inside the function, I instantiated an object from the client class. The client has so many functions that can be used based on your application requirement.
You can refer to the Class witty page for more information here
Since I wanted to do post request that contains a file (eg.Verilog file) as an uploaded file. I really needed to understand the format of the HTTP request first.

HTTP Request:

The HTTP request can be divided into three main divisions:

• A Request-line (Which contains the some info about the request such as the method of the request (post, get, put ..etc) , the url and the version of the HTTP protocol)

• The headers (The general header , the request header and finally the entity header)

• An empty line

• The message (Contains the file that needed to be uploaded to other server). (Optional)

By using the Message class, you can set the headers and add the message you want to send using the functions provided by the Message class such as:

• setHeader (const std::string &name, const std::string &value)

• addHeader (const std::string &name, const std::string &value)

• addBodyText (const std::string &text)

But you should really be careful when you add the message body text as you have to always follow the HTTP protocols in order to get your message interpreted by the receiver side.Refer to the HTTP RFC for more information

After getting the request ready (Header and the message body (contains the content of the file (e.g verilog file)) ) I used the Client class object function “Post” to the post request.
The function takes both the url and the request message as parameters. I managed at the end to get the request sent and then received and interpreted by the other engine. I encountered some problems regarding the message format so I had to go back and forth reading about the HTTP protocol until I get it done.

Some Cool stuff:

At the beginning I was curious to look at the full request that usually sent by the browser to the server when we create simple HTML form and then upload certain file to that server.
I was looking for a way to get that request. After a quick search on Google I encountered someone who posted a shell command that helps you to listen to certain port.
The code that I ran on the terminal was as follows:

while true; do printf '' | nc -l localhost xxxx ; done

I didn’t get exactly the full picture of the “nc” command. It seems that there’s yet a lot of things to discover. Any way I was later told by my co-worker Islam that I can access the request and responses messages directly from the chrome browser if I want. You can use either way of course when you want to play with HTTP .

That is all for this week. If you have any question just post them in the comment section below .. I’ll really be happy to assist you 🙂 SEE YOU NEXT WEEK