14. Internet Scripting

 

 

 

 

 

 

Topics

 

 

     Sockets in Python

     The FTP module (client)

     Email processing (client)

     CGI scripts (server)

     Jython: Python for Java systems

     Active Scripting and COM

     Other tools: urllib, web frameworks, XML (see Text), and more

 

 


 

Using sockets in Python

 

 

 

     Basic network and local client/server communication

     At the heart of Internet communications

     A standard C extension type: BSD socket wrapper

     C functions become socket object methods

 

Socket module details

 

     Supports all types: TCP/IP, UDP datagram, Unix domain

     Also supports timeouts, non-blocking, SSL

     Encrypted sockets supported if SSL library enabled

     Urllib modules do https:// if secure sockets enabled

     Email also uses secure sockets if enabled

     Python also supports “select” multiplex processing

 

 

 

 

Basic client/server example

 

      Server: echoes all data that it receives back

      Client: sends data to the server

      Client calls: “socket”, “connect”

      Server calls: “socket”, “bind”, “listen”, “accept”

 

 

 

 

file: echoserver.py

#########################################################

# Server side: open a socket on a port, listen for

# a message from a client, and send an echo reply;

# this is a simple one-shot listen/reply per client,

# but it goes into an infinite loop to listen for

# more clients as long as this server script runs;

#########################################################

 

from socket import *                    # get socket constructor and constants

myHost = ''                             # server machine, '' means local host

myPort = 50007                          # listen on a non-reserved port number

 

sockobj = socket(AF_INET, SOCK_STREAM)       # make a TCP socket object

sockobj.bind((myHost, myPort))               # bind it to server port number

sockobj.listen(5)                            # listen, allow 5 pending connects

 

while 1:                                     # listen until process killed

    connection, address = sockobj.accept()   # wait for next client connect

    print 'Server connected by', address     # connection is a new socket

    while 1:

        data = connection.recv(1024)         # read next line on client socket

        if not data: break                   # send a reply line to the client

        connection.send('Echo=>' + data)     # until eof when socket closed

    connection.close()

 

 

 

 

file: echoclient.py

#############################################################

# Client side: use sockets to send data to the server, and

# print server's reply to each message line; 'localhost'

# means that the server is running on the same machine as

# the client, which lets us test client and server on one

# machine;  to test over the net, run server on a remote

# machine, set serverHost to machine's domain name or IP addr;

#############################################################

 

import sys

from socket import *              # portable socket interface plus constants

serverHost = 'localhost'          # server name, or: 'starship.python.net'

serverPort = 50007                # non-reserved port used by the server

 

message = ['Hello network world']           # text to send to server

sockobj = socket(AF_INET, SOCK_STREAM)      # make a TCP/IP socket object

sockobj.connect((serverHost, serverPort))   # connect to serve and port

 

for line in message:

    sockobj.send(line)                      # send line to server over socket

    data = sockobj.recv(1024)               # receive from server: up to 1k

    print 'Client received:', `data`

 

sockobj.close()                             # close to send eof to server

 

 

 

 

 

 

Forking socket servers

 

import os, time, sys, signal, signal

from socket import *                   # get socket constructor and constants

myHost = ''                            # server machine, '' means local host

myPort = 50007                         # listen on a non-reserved port number

 

sockobj = socket(AF_INET, SOCK_STREAM)           # make a TCP socket object

sockobj.bind((myHost, myPort))                   # bind to server port number

sockobj.listen(5)                                # up to 5 pending connects

signal.signal(signal.SIGCHLD, signal.SIG_IGN)    # avoid child zombie processes

 

def now():                                       # time on server machine

    return time.ctime(time.time())

 

def handleClient(connection):                    # child process replies, exits

    time.sleep(5)                                # simulate a blocking activity

    while 1:                                     # read, write a client socket

        data = connection.recv(1024)

        if not data: break 

        connection.send('Echo=>%s at %s' % (data, now()))

    connection.close()

    os._exit(0)

 

def dispatcher():                                # listen until process killed

    while 1:                                     # wait for next connection,

        connection, address = sockobj.accept()   # pass to process for service

        print 'Server connected by', address,

        print 'at', now()

        childPid = os.fork()                     # copy this process

        if childPid == 0:                        # if in child process: handle

            handleClient(connection)             # else: go accept next connect

 

dispatcher()

 

 

 

 

 

 

Threading socket servers

 

import thread, time

from socket import *                  # get socket constructor and constants

myHost = ''                           # server machine, '' means local host

myPort = 50007                        # listen on a non-reserved port number

 

sockobj = socket(AF_INET, SOCK_STREAM)           # make a TCP/IP socket object

sockobj.bind((myHost, myPort))                   # bind to server port number

sockobj.listen(5)                                # upto 5 pending connects

 

def now():

    return time.ctime(time.time())               # current time on the server

 

def handleClient(connection):                    # in spawned thread: reply

    time.sleep(5)                                # simulate a blocking activity

    while 1:                                     # read, write a client socket

        data = connection.recv(1024)

        if not data: break

        connection.send('Echo=>%s at %s' % (data, now()))

    connection.close()

 

def dispatcher():                                # listen until process killd

    while 1:                                     # wait for next connection,

        connection, address = sockobj.accept()   # pass to thread for service

        print 'Server connected by', address,

        print 'at', now()

        thread.start_new(handleClient, (connection,))

 

dispatcher()

 

 

 

 

 

 

Select socket servers

 

# event loop: listen and multiplex until server process killed

from select import select

print 'select-server loop starting'

while 1:

    readables, writeables, exceptions = select(readsocks, writesocks, [])

    for sockobj in readables:

        if sockobj in mainsocks:                     # for ready input sockets

            # port socket: accept new client

            newsock, address = sockobj.accept()      # accept should not block

            print 'Connect:', address, id(newsock)   # newsock is a new socket

            readsocks.append(newsock)                # add to select list, wait

        else:

            # client socket: read next line

            data = sockobj.recv(1024)                # recv should not block

            print '\tgot', data, 'on', id(sockobj)

            if not data:                             # if closed by the clients

                sockobj.close()                      # close here and remv from

                readsocks.remove(sockobj)            # del list else reselected

            else:

                # this may block: should really select for writes too

                sockobj.send('Echo=>%s at %s' % (data, now()))

 

 

 

 

 

 

Standard library server types

 

    Threading/Forking TCP/UDP server classes

    Asyncore select-based server classes

    See also: HTTP/CGI webserver.py in CGI section below

    See also: Twisted 3rd party system

    See also: asyncio module in 3.4+, async/await syntax in 3.5+

 

import SocketServer, time               # get socket server, handler objects

myHost = ''                             # server machine, '' means local host

myPort = 50007                          # listen on a non-reserved port number

def now():

    return time.ctime(time.time())

 

class MyClientHandler(SocketServer.BaseRequestHandler):

    def handle(self):                           # on each client connect

        print self.client_address, now()        # show this client's addr

        time.sleep(5)                           # simulate blocking activity

        while 1:                                # self.request is client socket

            data = self.request.recv(1024)      # read, write a client socket

            if not data: break

            self.request.send('Echo=>%s at %s' % (data, now()))

        self.request.close()

 

# make a threaded server, listen/handle clients forever

myaddr = (myHost, myPort)

server = SocketServer.ThreadingTCPServer(myaddr, MyClientHandler)

server.serve_forever()     

 

 

 

 

 

The FTP module

 

 

     ftplib library module uses sockets to transfer files

     Supports both binary and text retrieve/store operations

     Handles all handshaking with the remote site

     See Python library manual for more details

 

 

 

 

Example: fetch file

 

 

file: getone.py

import os, sys

from getpass import getpass

 

nonpassive  = False                     

filename    = 'lawnlake2-jan-03.jpg'          # file to download

dirname     = '.'                             # remote directory

sitename    = 'ftp.rmi.net'                   # ftp site to contact

userinfo    = ('lutz', getpass('Pswd?'))      # use () for anonymous

if len(sys.argv) > 1: filename = sys.argv[1]  # file on command-line?

 

print 'Connecting...'

from ftplib import FTP                      # socket-based ftp tools

localfile  = open(filename, 'wb')           # local file to store to

connection = FTP(sitename)                  # connect to ftp site

connection.login(*userinfo)                 # default anonymous login

connection.cwd(dirname)                     # xfer 1k at a time

if nonpassive:                              # if server requires

    connection.set_pasv(False)

   

# thread me in a GUI

print 'Downloading...'

connection.retrbinary('RETR ' + filename, localfile.write, 1024)

connection.quit()

localfile.close()

 

 

 

 

 

Example: simple FTP site mirror (PP book)

 

#!/bin/env python

########################################################################

# use ftp to copy all files from a remote site/directory to a local dir;

# e.g., run me periodically from a unix cron job to mirror an ftp site;

# script assumes this is a flat directory--see the mirror program in

# Python's Tools directory for a version that handles subdirectories;

########################################################################

 

import os, sys, ftplib

from getpass import getpass

 

remotesite = 'home.rmi.net'

remotedir  = 'public_html'

remoteuser = 'lutz'

remotepass = getpass('Please enter password for %s: ' % remotesite)

localdir   = (len(sys.argv) > 1 and sys.argv[1]) or '.'

cleanall   = raw_input('Clean local directory first? ')[:1] in ['y', 'Y']

 

print 'connecting...'

connection = ftplib.FTP(remotesite)                 # connect to ftp site

connection.login(remoteuser, remotepass)            # login as user/password

connection.cwd(remotedir)                           # cd to directory to copy

 

if cleanall:

    for localname in os.listdir(localdir):          # try to delete all locals

        try:                                        # to remove old files

            print 'deleting local', localname

            os.remove(os.path.join(localdir, localname))

        except:

            print 'cannot delete local', localname

 

count = 0                                           # download remote files

remotefiles = connection.nlst()                     # nlst() gives files list

                                                    # dir()  gives all details

for remotename in remotefiles:

    localname = os.path.join(localdir, remotename)

    print 'copying', remotename, 'to', localname

    if remotename[-4:] == 'html' or remotename[-3:] == 'txt':

        # use ascii mode xfer

        localfile = open(localname, 'w')

        callback  = lambda line, file=localfile: file.write(line + '\n')

        connection.retrlines('RETR ' + remotename, callback)

    else:

        # use binary mode xfer

        localfile = open(localname, 'wb')

        connection.retrbinary('RETR ' + remotename, localfile.write)

    localfile.close()

    count = count+1

 

connection.quit()

print 'Done:', count, 'files downloaded.'

 

 


Example: upload site by FTP (PP book)

 

#!/bin/env python

########################################################################

# use ftp to upload all files from a local dir to a remote site/directory;

# e.g., run me to copy an ftp site's files from your machine to your ISP,

# especially handy if you only have ftp access to your website, not a

# telnet/shell account access (else you could tar up all files and

# transfer in a single step to the remote machine and untar there);

# to upload subdirectories too, use os.path.isdir(path), FTP().mkd(path),

# and recursion--see uploadall.py for a version that supports subdirs.

##########################################################################

 

import os, sys, ftplib, getpass

 

remotesite = 'starship.python.net'                  # upload to starship site

remotedir  = 'public_html/home'                     # from win laptop or other

remoteuser = 'lutz'

remotepass = getpass.getpass('Please enter password for %s: ' % remotesite)

localdir   = (len(sys.argv) > 1 and sys.argv[1]) or '.'

cleanall   = raw_input('Clean remote directory first? ')[:1] in ['y', 'Y']

 

print 'connecting...'

connection = ftplib.FTP(remotesite)                 # connect to ftp site

connection.login(remoteuser, remotepass)            # login as user/password

connection.cwd(remotedir)                           # cd to directory to copy

 

if cleanall:

    for remotename in connection.nlst():            # try to delete remotes

        try:                                        # to remove old files

            print 'deleting remote', remotename

            connection.delete(remotename)          

        except:                                    

            print 'cannot delete remote', remotename

 

count = 0

localfiles = os.listdir(localdir)                   # upload all local files

                                                    # listdir() strips dirpath

for localname in localfiles: 

    localpath = os.path.join(localdir, localname)

    print 'uploading', localpath, 'to', localname

    if localname[-4:] == 'html' or localname[-3:] == 'txt':

        # use ascii mode xfer

        localfile = open(localpath, 'r')

        connection.storlines('STOR ' + localname, localfile)

    else:

        # use binary mode xfer

        localfile = open(localpath, 'rb')

        connection.storbinary('STOR ' + localname, localfile, 1024)

    localfile.close()

    count = count+1

 

connection.quit()

print 'Done:', count, 'files uploaded.'


 

 

 

 

 

Email processing

 

 

     POP, IMAP (retrieve); SMTP (send); on top of sockets

     Newer email.* module package: parse+compose, attachments, MME en/decoding …

 

 

 

 

Reading a POP email account

 

#!/usr/local/bin/python

######################################################

# use the Python POP3 mail interface module to view

# your pop email account messages;

######################################################

 

import poplib, getpass, sys, mailconfig

 

mailserver = mailconfig.popservername      # e.g., 'pop.rmi.net'

mailuser   = mailconfig.popusername        # e.g., 'lutz'

mailpasswd = getpass.getpass('Password for %s?' % mailserver)

 

print 'Connecting...'

server = poplib.POP3(mailserver)

server.user(mailuser)                      # connect,login to server

server.pass_(mailpasswd)                   # pass is a reserved word

try:

    print server.getwelcome()              # print greeting message

 

    msgCount, msgBytes = server.stat()

    print 'There are', msgCount, 'mail messages in', msgBytes, 'bytes'

    print '-'*80

    raw_input('[Press Enter key]')

 

    for i in range(msgCount):

        hdr, message, octets = server.retr(i+1)

        for line in message: print line          # retrieve, print all

        print '-'*80                             # mbox locked till quit

        if i < msgCount - 1:

           raw_input('[Press Enter key]')

finally:

    server.quit()                                  # make sure to unlock

print 'Bye.'

 


Sending email via a SMTP server

 

 

#!/usr/local/bin/python

########################################################

# use the Python SMTP mail interface module to send mail

########################################################

 

import smtplib, sys, time, mailconfig       # oct15: dropped string

mailserver = mailconfig.smtpservername      # e.g., starship.python.net

 

From = raw_input('From? ').strip()          # e.g, lutz@rmi.net

To   = raw_input('To?   ').strip()          # e.g., spam@python.org

To   = To.split(',')                        # allow a list of tos

Subj = raw_input('Subj? ').strip()

 

# prepend standard headers (this may be nonstandard)

date = time.ctime(time.time())

text = ('From: %s\nTo: %s\nDate: %s\nSubject: %s\n'

                         % (From, string.join(To, ','), date, Subj))

text = text + '\n'   # blank line between hdrs,body

 

print 'Type message text, end with line=(ctrl + D or Z)'

while True:

    line = sys.stdin.readline()

    if not line:

        break                                  # exit on ctrl-d

    text += line                               # servers do this auto

 

if sys.platform[:3] == 'win': print

print 'Connecting...'

server = smtplib.SMTP(mailserver)              # connect, no login step

errors = server.sendmail(From, To, text)

server.quit()

if errors:                                     # smtplib also raises excepts

    print 'Errors:', errors

else:

    print 'No errors.'

print 'Bye.'

 

 

 

 

The email package

    Parses mail text fetched with poplib

    Generates mail text to send with smtplib

 

# Composing a simple message

 

>>> from email.Message import Message

>>> m = Message()

>>> m['from'] = 'Sue Jones <sue@jones.com>'

>>> m['to']   = 'pp3e@earthlink.net'

>>> m.set_payload('The owls are not what they seem...')

>>> s = str(m)

>>> print s

From nobody Sun Jan 22 21:26:53 2006

from: Sue Jones <sue@jones.com>

to: pp3e@earthlink.net

 

The owls are not what they seem...

 

 

# Parsing a simple message

 

>>> from email.Parser import Parser

>>> x = Parser().parsestr(s)

>>> x

<email.Message.Message instance at 0x00A7DA30>

>>> x['From']

'Sue Jones <sue@jones.com>'

>>> x.get_payload()

'The owls are not what they seem...'

>>> x.items()

[('from', 'Sue Jones <sue@jones.com>'), ('to', 'pp3e@earthlink.net')]

 

 

>>> for part in x.walk():

...     print x.get_content_type()

...     print x.get_payload()

...

text/plain

The owls are not what they seem...

 

 

 

# Composing a multi-part message (attachments)

>>> from email.MIMEMultipart import MIMEMultipart

>>> from email.MIMEText import MIMEText

>>> 

>>> top = MIMEMultipart()

>>> top['from'] = 'Art <arthur@camelot.org>'

>>> top['to']   = 'pp3e@earthlink.net'

>>> 

>>> sub1 = MIMEText('nice red uniforms...\n')

>>> sub2 = MIMEText(open('data.txt').read())

>>> sub2.add_header('Content-Disposition', 'attachment',

 filename='data.txt')

>>> top.attach(sub1)

>>> top.attach(sub2)

 

>>> text = top.as_string()    # same as str() or print

>>> print text

Content-Type: multipart/mixed; boundary="===============0257358049=="

MIME-Version: 1.0

from: Art <arthur@camelot.org>

to: pp3e@earthlink.net

 

--===============0257358049==

Content-Type: text/plain; charset="us-ascii"

MIME-Version: 1.0

Content-Transfer-Encoding: 7bit

 

nice red uniforms...

 

--===============0257358049==

Content-Type: text/plain; charset="us-ascii"

MIME-Version: 1.0

Content-Transfer-Encoding: 7bit

Content-Disposition: attachment; filename="data.txt"

 

line1

line2

line3

 

--===============0257358049==--

 

 

# Parsing a multi-part message

 

>>> from email.Parser import Parser

>>> msg = Parser().parsestr(text)

>>> msg['from']

'Art <arthur@camelot.org>'

 

>>> for part in msg.walk():

...     print part.get_content_type()

...     print part.get_payload()

...     print

...

multipart/mixed

[<email.Message.Message instance at 0x00A82058>,

<email.Message.Message instance at 0x00A82260>]

 

text/plain

nice red uniforms...

 

 

text/plain

line1

line2

line3

 


Running the email examples

 

 

    See also: pymail.py and PyMailGui.py in examples directory

    Also in Extras, PP4E (book examples)

    PyMailGui is a Python/Tk mail client (screen shots in GUI unit)

 

 

C:\examples\Part3\Internet\Email>python smtpmail.py

From? lutz@rmi.net

To?   lutz@rmi.net

Subj? testing 1 2 3

Type message text, end with line=(ctrl + D or Z)

words

Connecting...

No errors.

Bye.

 

 

C:\examples\Part3\Internet\Email>python popmail.py

Password for pop.rmi.net?

Connecting...

+OK Cubic Circle's v1.31 1998/05/13 POP3 ready

There are 1 mail messages in 780 bytes

-----------------------------------------------------------------------

 

 

[Press Enter key]

Received: by chevalier (mbox lutz)

 (with Cubic Circle's cucipop (v1.31 1998/05/13) Sun Feb 13 13:56:44 2000)

X-From_: lutz@rmi.net  Sun Feb 13 13:43:01 2000

Return-Path: <lutz@chevalier.rmi.net>

Received: from server.python.net (server.python.net [209.50.192.113])

        by chevalier.rmi.net (8.9.3/8.9.3) with SMTP id NAA16859

        for <lutz@rmi.net>; Sun, 13 Feb 2000 13:43:00 -0700 (MST)

Message-Id: <200002132043.NAA16859@chevalier.rmi.net>

Received: (qmail 31944 invoked from network); 13 Feb 2000 20:43:22 -0000

Received: from dial-67.73.denco.rmi.net (166.93.67.73)

  by server.python.net with SMTP; 13 Feb 2000 20:43:22 -0000

From: lutz@rmi.net

To: lutz@rmi.net

Date: Sun Feb 13 13:44:16 2000

Subject: testing 1 2 3

 

words

 

------------------------------------------------------------------------

 

Bye.

 

 

 

Other client-side tools


 

 

 

      urllib: fetching web pages

 

      nntplib: reading and posting usenet news

 

      httplib: lower-level web conversations

 

      telnet, gopher, imap,…

 

      htmllib, xml package: parsing fetched web pages and data (soap)

 

      Active Scripting: embedded Python in HTML

 

      Jython: client-side applets in Python

 

      RIAs: pyjamas, Silverlight, … (see GUIs section)

 

 

 

 

Building web sites with Python

 

 

 

     CGI scripting

      Simpler techniques for simpler sites

      Augment with state retention: hidden form fields, query parameters, cookies, server-side databases

      Embeds HTML in Python (opposite of PSP, etc.)

      Cgi module parses inputs, escapes outputs

      See also HTMLgen and similar for reply generation

      Require web server that can run Python scripts

 

     Web frameworks and tools

      Zope (OO meets websites: DTML, TAL, ZMI, ZODB, …)

      Plone (CMS built on top of Zope, workflow)

      TurboGears (multi-tool package: AJAX, MVC, SQLObject, CherryPy,…)

      Django (model view controller, implicit)

      CherryPy, WebWare, Quixote, Pylons, web2py (embarrassment of riches!)

      PSP (Webware and mod_python, like PHP/ASP)

      mod_python for Apache (speed, sessions, PSP,…)

      Fast CGI (persistent processes for state retention)

      Google AppEngine, Amazon AWS, Microsoft Azure

      Twisted (network server framework)

 

 

 

Writing server-side CGI scripts

 

 

     Server-side scripts referenced from HTML pages

     Run on server, connected to client/browser via sockets

     cgi module handles input parsing, output formatting

 

 

Typical operation

 

 

     Inputs: fetch info typed into forms  (cgi.FieldStorage)

      cgi scripts get input from stdin plus environment info

     cgi library module

      provides dictionary interface to parsed form data

     Outputs: sesults in browser  (cgi.escape, urllib.quote)

      cgi scripts write HTML to stdout to display results

 

 

 

 


 

Basic CGI interfaces

 

 

 

     FieldStorage class

      Reads the form contents from standard input or the environment

 

 

 

print "Content-type: text/html"   # HTML text follows

print                             # blank line: end headers

print "<TITLE>CGI script output</TITLE>"

 

form = cgi.FieldStorage()

form_ok = 0

if form.has_key("name") and form.has_key("addr"):

    if (form["name"].value != "" and

        form["addr"].value != ""):

        form_ok = 1

 

if not form_ok:

    print "<H1>Error</H1>"

    print "Please fill in the name and addr fields."

    return

...more form processing here...

 

 

 

 

     FormContent class

      A (now) outdated interface: same functionality, but FieldStorage above is preferred scheme

 

 

 

import cgi                   # see library manual

. . .

form = cgi.FormContent()     # parse input stream

if form.has_key("fieldname"):

    data = form["fieldname"][0]


 

 

 

 

Basic CGI example

 

 

 

 

# running a local webserver: Extras\Code\Internet\webserver.py

 

#!/usr/bin/python3

"""

===========================================================================

Implement an HTTP web server in Python that knows how to run server-side

CGI scripts coded in Python;  serves files and scripts from current working

dir;  Python scripts must be stored in webdir\cgi-bin or webdir\htbin;

===========================================================================

"""

 

import os, sys

from http.server import HTTPServer, CGIHTTPRequestHandler

 

webdir = '.'   # where your html files and cgi-bin script directory live

port   = 80    # default http://localhost/, else use http://localhost:xxxx/

 

os.chdir(webdir)                                       # run in HTML root dir

srvraddr = ("", port)                                  # my hostname, portnumber

srvrobj  = HTTPServer(srvraddr, CGIHTTPRequestHandler)

srvrobj.serve_forever()                                # run as perpetual daemon

 

 

 

# input form: cgi101.html (use GET instead of POST for some server/browser combos)

 

<html>

<title>Interactive Page</title>

<body>

<form method=POST action="cgi-bin/cgi101.py">

    <P><B>Enter your name:</B>

    <P><input type=text name=user>

    <P><input type=submit>

</form>

</body></html>

 

 

 

 

 

 

# reply script: cgi-bin/cgi101.py

 

#!/usr/bin/python3

import cgi

form = cgi.FieldStorage()                 # parse form data

print('Content-type: text/html\n')        # hdr plus blank line

print('<title>Reply Page</title>')        # html reply page

if not 'user' in form:

    print('<h1>Who are you?</h1>')

else:

    print('<h1>Hello <i>%s</i>!</h1>' % cgi.escape(form['user'].value))

 

 

 

 

 

 

 

# testing with query strings and urllib (see also: htmllib parser)

 

 

>>> from urllib.request import urlopen       # 2.X: from urllib import…

 

>>> conn = urlopen('http://localhost/cgi-bin/cgi101.py?user=Sue+Smith')

>>> reply = conn.read()

>>> reply

b'<title>Reply Page</title>\r\n<h1>Hello <i>Sue Smith</i>!</h1>\r\n'

 

>>> urlopen('http://localhost/cgi-bin/cgi101.py').read()

b'<title>Reply Page</title>\r\n<h1>Who are you?</h1>\r\n'

 

>>> urlopen('http://localhost/cgi-bin/cgi101.py?user=Bob').read()

b'<title>Reply Page</title>\r\n<h1>Hello <i>Bob</i>!</h1>\r\n'

 

 

 

 

 

 

CGI controls: test5.html/.cgi (PP book)

 

 

test5b.html

<HTML><BODY>

<TITLE>CGI 101</TITLE>

<H1>Common input devices: alternative layout</H1>

<P>Use the same test5.cgi server side script, but change the

layout of the form itself.  Notice the separation of user interface

and processing logic here; the CGI script is independent of the

HTML used to interact with the user/client.</P><HR>

 

<FORM method=POST action="http://starship.python.net/~lutz/test5.cgi">

  <H3>Please complete the following form and click Submit</H3>

  <P><TABLE border cellpadding=3>

    <TR>

      <TH align=right>Name:

      <TD><input type=text name=name>

    <TR>

      <TH align=right>Shoe size:

      <TD><input type=radio name=shoesize value=small>Small

          <input type=radio name=shoesize value=medium>Medium

          <input type=radio name=shoesize value=large>Large

    <TR>

      <TH align=right>Occupation:

      <TD><select name=job>

        <option>Developer

        <option>Manager

        <option>Student

        <option>Evangelist

        <option>Other

      </select>

    <TR>

      <TH align=right>Political affiliations:

      <TD><P><input type=checkbox name=language value=Python>Pythonista

          <P><input type=checkbox name=language value=Perl>Perlmonger

          <P><input type=checkbox name=language value=Tcl>Tcler

    <TR>

      <TH align=right>Comments:

      <TD><textarea name=comment cols=30 rows=2>Enter spam here</textarea>

    <TR>

      <TD colspan=2 align=center>

      <input type=submit value="Submit">

      <input type=reset  value="Reset">

  </TABLE>

</FORM>

</BODY></HTML>

 

 

 

 

 

 

 

 

test5.cgi

#!/usr/local/bin/python

# runs on the server, reads form input, prints html;

# executable privileges, stored in ~/public_html,

 

import cgi, sys, string

form = cgi.FieldStorage()            # parse form data

print "Content-type: text/html"      # plus blank line

 

html = """

<TITLE>test4.cgi</TITLE>

<H1>Greetings</H1>

<HR>

<H4>Your name is %(name)s</H4>

<H4>You wear rather %(shoesize)s shoes</H4>

<H4>Your current job: %(job)s</H4>

<H4>You program in %(language)s</H4>

<H4>You also said:</H4>

<P>%(comment)s</P>

<HR>"""

 

data = {}

for field in ['name', 'shoesize', 'job', 'language', 'comment']:

    if not form.has_key(field):

        data[field] = '(unknown)'

    else:

        if type(form[field]) != type([]):

            data[field] = form[field].value

        else:

            values = map(lambda x: x.value, form[field])

            data[field] = string.join(values, ' and ')

print html % data

 

 

 

 

 

 

 

In typical interactions, a user directs a browser to a HTML file, which includes a form action that automatically invokes the server-side CGI script on submit button press.  It’s also possible to invoke a CGI script directly, provided that input fields are given values within the referencing URL address itself:

 

test5c.html

<HTML><BODY>

<TITLE>CGI 101</TITLE>

<H1>Common input devices: URL parameters</H1>

 

<P>

This demo invokes the test5.cgi server-side script again,

but hardcodes input data to the end of the script's URL,

within a simple hyperlink (instead of packaging up a form's

inputs).  Click your browser's "show page source" button

to view the links associated with each list item below.

 

<P>This is really more about CGI than Python, but notice that

Python's cgi module handles both this form of input (which is

also produced by GET form actions), as well as POST-ed forms;

they look the same to the Python CGI script.  In other words,

cgi module users are independent of the method used to submit

data.

 

<P>Also notice that URLs with appended input values like this

can be generated as part of the page output by another CGI script,

to direct a next user click to the right place and context; together

with type 'hidden' input fields, they provide one way to

save state between clicks.

</P><HR>

 

<UL>

<LI><A href=

"http://starship.python.net/~lutz/test5.cgi?name=Bob&shoesize=small">

Send Bob, small</A>

 

<LI><A href=

"http://starship.python.net/~lutz/test5.cgi?name=Tom&language=Python">

Send Tom, Python</A>

 

<LI><A href=

"http://starship.python.net/~lutz/test5.cgi?job=Evangelist&comment=spam">

Send Evangelist, spam</A>

</UL>

 

</UL>

<HR>

</BODY></HTML>

 

 

 

 

 

The Grail web browser (educational only today)

    

      Dated but educational: a contemporary of Netscape!

      Portable: written in Python, uses Tkinter as browser GUI

      Browser downloads/runs applets referenced in HTML

      Applets more powerful than HTML+CGI: a full GUI API

      Python also supports ActiveX, Netscape plug-ins,…

 

HTML file

<HEAD>

<TITLE>Grail Applet Test Page</TITLE>

</HEAD>

<BODY>

<H1>Test an Applet Here!</H1>

Click this button!

<APP CLASS=Question>

</BODY>

 

 

file: Question.py

# Python applet file: Question.py

# in the same location (URL) as the html file

# that references it; adds widgets to browser;

 

from Tkinter import *

 

class Question:                          # run by grail?

    def __init__(self, parent):          # parent=browser

        self.button = Button(parent,

                             bitmap='question',

                             command=self.action)

        self.button.pack()

 

    def action(self):

        if self.button['bitmap'] == 'question':

            self.button.config(bitmap='questhead')

        else:

            self.button.config(bitmap='question')

 

if __name__ == '__main__':

    root = Tk()                    # run stand-alone?

    button = Question(root)        # parent=Tk: top-level

    root.mainloop()


 

 

 

Grail sample screen: a Python/Tk GUI

 

Caveat: this is a very old example: [tT[kinter looks a lot better today!

 

 

Description: Description: Description: Description: Description: Description: Description: Description: grail

  

 

 

 


 

Jython: Python for Java systems

 

 

     An independent implementation of Python

     Python for Java-based applications

     Python scripting complements Java systems language

     Renamed from “JPython” to “Jython” (2000: copyright)

     IronPython is very similar, but for .Net/C#

 

 

Structure

 

 

      A collection of Java classes that compile and run Python code

      jython” program: equivalent to “python”, interactive, scripts

      jythonc” compiler/packager program: makes .jar, .class files

      Embedding: “PythonInterpreter” class runs Python from Java

      Extending: automatic, Python coded imports, uses Java classes

 

 

 

Advantages

 

 

     Compiles Python code to JVM byte-code

      “100% Pure Java” (written in Java, generates JVM)

      More seamless than Tcl and Perl approaches

      Can run as client-side applets in Java-aware browsers

 

     Includes Python/Java integration support

      Python can access Java classes

      Java can embed/call Python scripts

      Access to AWT and Swing from Python for GUIs

 


 

 

Examples

 

 

 

Hello World

from java.applet import Applet

 

class HelloWorld(Applet):

    def paint(self, gc):

        gc.drawString("Hello world", 20, 30)

 

 

 

A (very!) simplistic calculator (see PP’s PyCalc)

 

from java import awt

from pawt import swing

 

labels = ['7', '8', '9', '+',

          '4', '5', '6', '-',

          '1', '2', '3', '*',

          '0', '.', '=', '/' ]

 

keys = swing.JPanel(awt.GridLayout(4, 4))

display = swing.JTextField()

 

def push(event):                # Callback for regular keys

    display.replaceSelection(event.actionCommand)

 

def enter(event):               # Callback for '=' key

    display.text = str(eval(display.text))

    display.selectAll()

 

for label in labels:

    key = swing.JButton(label)

    if label == '=':

        key.actionPerformed = enter

    else:

        key.actionPerformed = push

    keys.add(key)

 

panel = swing.JPanel(awt.BorderLayout())

panel.add("North", display)

panel.add("Center", keys)

swing.test(panel)


 

 

 

Description: Description: Description: Description: Description: Description: Description: Description: awtsmall

 

 

 

 

 

Description: Description: Description: Description: Description: Description: Description: Description: awtcalc

 

 

 

 

Jython downsides

 

     Not yet fully compatible with standard Python

     Only applicable where a JVM is installed or shipped

     Requires users to learn Java too (environment, libs)

     Doesn’t support standard C/C++ extension modules

     Slower than C Python (1.7x, 10x, 100X), for now?

 

Active Scripting and COM

 

    Active Scripting: Python embedded in HTML, like JavaScript or VBScript

    Extracted and run on client (browser object module) or server (form and reply object models)

    Windows only, IE and IIS only today, requires Python + PyWin32 (formerly win32all) package installs

    More useful on server side: only one machine requires Python + PyWin32 installs, any browser will work

    COM and DCOM: Microsoft component object models; DCOM allows client and server to be remote

    Host uses Active Scripting to export COM object interfaces for use in Python code

    See also Python Soap, XML-RPC support for web services

    More recent: IronPython Python port to C#/.Net framework, xlwings Excel plugin

 

Active Scripting example

 

Embedded Python in HTML was disabled on client due to rexec security issue, though Javascript can still call registered Python COM objects there; see also Pyjs, IronPython, ASP, PSP, Zope,…

 

<HTML>

<BODY>

<H1>Embedded code demo: Python</H1>

<SCRIPT Language=Python>

 

# embedded python code shows three alert boxes

# as page is loaded; any Python code works here,

# and uses auto-imported global funcs and objects

 

def message(i):

    if i == 2:

        alert("Finished!")

    else:

        alert("A Python-generated alert => %d" % i)

 

for count in range(3): message(count)

 

</SCRIPT>

</BODY></HTML>

 

 

A Python COM client

from sys import argv

docdir = 'C:\\temp\\'

if len(argv) == 2: docdir = argv[1]              # ex: comclient.py a:\

 

from win32com.client import Dispatch             # early or late binding

word  = Dispatch('Word.Application')             # connect/start word

word.Visible = 1                                 # else word runs hidden

 

# create and save new doc file

newdoc = word.Documents.Add()                    # call word methods

spot   = newdoc.Range(0,0)

spot.InsertBefore('Hello COM client world!')     # insert some text

newdoc.SaveAs(docdir + 'pycom.doc')              # save in doc file

newdoc.SaveAs(docdir + 'copy.doc')  

newdoc.Close()

 

 

 

A Python COM server

import sys

from   win32com.server.exception import COMException         # what to raise

import win32com.server.util                                  # server tools

globhellos = 0

 

class MyServer:

 

    # com info settings

    _reg_clsid_      = '{1BA63CC0-7CF8-11D4-98D8-BB74DD3DDE3C}'

    _reg_desc_       = 'Example Python Server'

    _reg_progid_     = 'PythonServers.MyServer'           # external name

    _reg_class_spec_ = 'comserver.MyServer'               # internal name

    _public_methods_ = ['Hello', 'Square']

    _public_attrs_   = ['version']

 

    # python methods

    def __init__(self): 

        self.version = 1.0

        self.hellos  = 0

    def Square(self, arg):                                # exported methods

        return arg ** 2

    def Hello(self):                                      # global variables

        global globhellos                                 # retain state, but

        globhellos  = globhellos  + 1                     # self vars don't

        self.hellos = self.hellos + 1

        return 'Hello COM server world [%d, %d]' % (globhellos, self.hellos)

 

# registration functions

def Register(pyclass=MyServer):

    from win32com.server.register import UseCommandLine

    UseCommandLine(pyclass)

def Unregister(classid=MyServer._reg_clsid_):

    from win32com.server.register import UnregisterServer

    UnregisterServer(classid)

 

if __name__ == '__main__':       # register server if file run or clicked

    Register()                   # unregisters if --unregister cmd-line arg

 

 

Other Internet-related tools

 

 

 

See Extras\Code\Internet directory on the class CD for more examples

 

 

     HTMLgen

      generates HTML from class-based page descriptions (see CD)

     HTML, SGML parsers

      useful for extracting information from web pages (standard lib, BeautifulSoup)

     rfc822 module, newer email package

      parses standard message headers (and much more)

     ILU, fnorb, and OmniORB packages

      can be used for CORBA networking (if anyone still does)

     urllib module

      opens URL, returns file-like object: read, readline (see CD)

      Parse fetched page with htmllib, string.find, xml package

     Restricted execution mode (security issue)

      See Extras\Code\Internet directory on CD

      for running code fetched from untrusted sources

      modules “rexec” and “bastion”

      Withdrawn in Python 2.4: security concerns

     Other Internet protocol support

      web server classes (see Extras\Code\Internet on CD)

      telnetlib, nntplib (news), SSL sockets,…

     Zope & Plone (www.zope.org)

      An open-source web application framework

      Written in and customized with Python

      Plone runs on Zope: workflow-based content management

      “http://server/module/func?arg1=val1&arg2=val2” URL becomes

module.func(arg1=val1, arg2=val2)” call on server-side Python

      [Defunct] Supplemental Zope/Plone workbook available

     Django: Python’s Rails?


 

 

 

Lab Session 10

 

Click here to go to lab exercises

Click here to go to exercise solutions

Click here to go to solution source files

 

Click here to go to lecture example files