How to set up Python server side with javascript client side

Answer #1:

a. To handle input/output of the program: Pexpect. It’s fairly easy to use, and reading some of the examples distributed with it should teach you enough to get the basics down.

b. Javascript interface:

Well, I use gevent and it’s built-in WSGI server. (look up what a WSGI server (another) is). I should note that this program will keep a state, so you can manage your open sessions by returning a session ID to your javascript client and storing your pexpect session in a global variable or some other container so that you can complete the program’s input and output across multiple independent AJAX requests. I leave that up to you, however, as that is not as simple.

All my example will do is put the POST request in some after clicking something of your choice. (it won’t actually work because some of the variables are not set. Set them.)

Heres the relevant parts:

<!-- JavaScript -->
<script src="jquery.js"></script>
<script type="text/javascript">
function toPython(usrdata){
    $.ajax({
        url: "http://yoursite.com:8080",
        type: "POST",
        data: { information : "You have a very nice website, sir." , userdata : usrdata },
        dataType: "json",
        success: function(data) {
            <!-- do something here -->
            $('#somediv').html(data);
        }});
$("#someButton").bind('click', toPython(something));
</script>

Then the server:

# Python and Gevent
from gevent.pywsgi import WSGIServer
from gevent import monkey
monkey.patch_all() # makes many blocking calls asynchronous

def application(environ, start_response):
    if environ["REQUEST_METHOD"]!="POST": # your JS uses post, so if it isn't post, it isn't you
        start_response("403 Forbidden", [("Content-Type", "text/html; charset=utf-8")])
        return "403 Forbidden"
    start_response("200 OK", [("Content-Type", "text/html; charset=utf-8")])
    r = environ["wsgi.input"].read() # get the post data
    return r

address = "youraddresshere", 8080
server = WSGIServer(address, application)
server.backlog = 256
server.serve_forever()

If your program is Object Oriented, it’d be fairly easy to integrate this. EDIT: Doesn’t need to be object oriented. and I have now included some Pexpect code

global d
d = someClass()
def application(environ, start_response):
    # get the instruction
    password = somethingfromwsgi # read the tutorials on WSGI to get the post stuff
    # figure out WHAT to do
    global d
    success = d.doSomething()
    # or success = funccall()
    prog = pexpect.spawn('python someprogram.py')
    prog.expect("Password: ")
    prog.sendline(password)
    i = prog.expect(["OK","not OK", "error"])
    if i==0:
        start_response("200 OK", [("Content-Type", "text/html; charset=utf-8")])
        return "Success"
    elif i==1:
        start_response("500 Internal Server Error", [("Content-Type", "text/html; charset=utf-8")])
        return "Failure"
    elif i==2:
        start_response("500 Internal Server Error", [("Content-Type", "text/html; charset=utf-8")])
        return "Error"

Another option I suggest is Nginx + uWSGI. If you would prefer that, I can give you some examples of that as well. It gives you the benefit of incorporating the webserver into the setup.

Answer #2:

To pass transparently your data from javascript to external Python program you could use WebSocket protocol to connect your server and javascript, and use stdin/stdout to communicate with the external program from the server.

Here’s an example Python program client.py:

#!/usr/bin/env python
"""Convert stdin to upper case."""
for line in iter(raw_input, 'quit'):
    print line.upper()

I’ve created a server using code from hello world websocket example and SO answer about how to create a new process on each incoming connection and to redirect all input data to the process’ stdin:

#!/usr/bin/python
"""WebSocket CLI interface."""
import sys
from twisted.application import strports # pip install twisted
from twisted.application import service
from twisted.internet    import protocol
from twisted.python      import log
from twisted.web.server  import Site
from twisted.web.static  import File

from txws import WebSocketFactory # pip install txws


class Protocol(protocol.Protocol):
    def connectionMade(self):
        from twisted.internet import reactor
        log.msg("launch a new process on each new connection")
        self.pp = ProcessProtocol()
        self.pp.factory = self
        reactor.spawnProcess(self.pp, sys.executable,
                             [sys.executable, '-u', 'client.py'])
    def dataReceived(self, data):
        log.msg("redirect received data to process' stdin: %r" % data)
        self.pp.transport.write(data)
    def connectionLost(self, reason):
        self.pp.transport.loseConnection()

    def _send(self, data):
        self.transport.write(data) # send back


class ProcessProtocol(protocol.ProcessProtocol):
    def connectionMade(self):
        log.msg("connectionMade")
    def outReceived(self, data):
        log.msg("send stdout back %r" % data)
        self._sendback(data)
    def errReceived(self, data):
        log.msg("send stderr back %r" % data)
        self._sendback(data)
    def processExited(self, reason):
        log.msg("processExited")
    def processEnded(self, reason):
        log.msg("processEnded")

    def _sendback(self, data):
        self.factory._send(data)


application = service.Application("ws-cli")

_echofactory = protocol.Factory()
_echofactory.protocol = Protocol
strports.service("tcp:8076:interface=127.0.0.1",
                 WebSocketFactory(_echofactory)).setServiceParent(application)

resource = File('.') # serve current directory INCLUDING *.py files
strports.service("tcp:8080:interface=127.0.0.1",
                 Site(resource)).setServiceParent(application)

The web-client part, sendkeys.html:

<!doctype html>
<title>Send keys using websocket and echo the response</title>
<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.7.0/jquery.min.js">
</script>
<script src="sendkeys.js"></script>
<input type=text id=entry value="type something">
<div id=output>Here you should see the typed text in UPPER case</div>

and sendkeys.js:

// send keys to websocket and echo the response
$(document).ready(function() {
    // create websocket
    if (! ("WebSocket" in window)) WebSocket = MozWebSocket; // firefox
    var socket = new WebSocket("ws://localhost:8076");

    // open the socket
    socket.onopen = function(event) {
    socket.send('connectedn');

    // show server response
    socket.onmessage = function(e) {
        $("#output").text(e.data);
    }

    // for each typed key send #entry's text to server
    $("#entry").keyup(function (e) {
        socket.send($("#entry").attr("value")+"n");
    });
    }
});

To try it:

• download this gist

• install twisted, txws:

  $ pip install twisted txws
  

• run:

  $ twistd -ny wscli.py
  

• visit http://localhost:8080/

• click on sendkeys.html and type something

Answer #3:

You probably want Flask, along with the json module.

Django is another option, but is probably too high-level for your needs.

Answer #4:

This depends on what sort of application you are wrapping and on how your GUI options translate into application commands. But you have two goals here:

1. Writing a wrapper to allow you to read your program’s output and provide it input.

2. Making a web-server to receive GUI events and convert them to commands to pass to your “wrapper”

I have done something like what you need to do.

1. Essentially you need to turn socket streams into discreet commands. The defacto tool for this is expect, and any of it’s wrappers (I’ve used pexpect, the Python wrapper, and had a good experience with it).

2. This part might not be simple. The problem is that your underlying program is persistently running, and so your web-server should be statefull to know about the program across requests. The other option is for your web-server to simply re-attach to the process and issue it commands, and send back a response when it is encountered in the stdout stream, but you might end up with long response times depending on how fast the program is. Also there is the mismatch that AJAX requests are asynchronous, while your underlying program is synchronous. So yes, this can get quite complex. It really depends on your program. If you could add some details about what the program and the GUI are like, it’d help.