Sharing My Notes

In the past couple of months I’ve started keeping my notes in a personal wiki using Vim Wiki. Rather than just keeping them on my drive, I decided to share what I’ve figured out. Right now most of what I have is links and a bit of sample code. When I do a major update I’ll post a link here. I hope you find it useful.

New Plans

I have a couple of announcements to make. First, I haven’t been able to produce anything substantial this week because I’ve had a severe respiratory bug that’s kept me out of commission since Monday. If the Doctor’s correct I should be up and rolling again in the next couple of days. Second, I’m going to be making some changes to the content of this blog. When I started most of my other projects had been lying fallow for months. Now three of them have picked up at once, all of which require that I learn specifically for them. This is great, it’s exciting, it means that I have to choose between them, the blog, and sleep.

Rather than drop the blog completely I’m going to be writing about what I learn as I’m working on each of these projects, right now the things in my queue are:

I’ve enjoyed working on learning new languages and sharing the projects as I’ve gone, and would like to continue to do so. So I intend to, either at a much slower schedule or intermittently as I have time, continue to do the new language project. I’d be interested in feedback from any current readers for what they’d like to see.

Smalltalk 1/4: Getting Started

This is the first article in a series, to read the whole series go here.

The Plan

You’ve heard about it, every child will have a portable computer, small enough to carry in her bookbag. Its software will be an open environment that they can change and program themselves. I’m talking, of course, about the Dynabook, from Alan Kay’s proposal in 1972. The programming language that was to be the center of the Dynabook and its environment was Smalltalk, one of the first object oriented programming languages. Smalltalk was intended to be easy for children to learn, to that end has a simple and highly regular syntax and a deeply interactive environment.

Dynabook was an idea before its time. It lives on in spirit in the OLPC project, which also contains Smalltalk, or an application of it in the form of Squeak’s EToys.

Modernly Smalltalk has a number of implementations. One of the best known is Squeak which Alan Kay continues to be involved with. Squeak is an wonderful programmer’s playground where Kay and his team have tried out a multitude of ideas, still with the goal of creating an environment that children may learn in. Unfortunately for the working programmer the playground has become rather cluttered with toys, so the Pharo team has made a stripped down version for a much leaner programming environment.

As I alluded to above, Smalltalk isn’t just a programming language, it’s an environment in its own right. When you open up the Pharo image you have an entire integrated development system, including the System Browser where you do most of your coding, the debugger and the unit test system. And all of which is written in Smalltalk and can be modified inside the system in real time.

What to Expect

When this article is complete:

  • You will have:
    • An installation of Pharo
    • An object that says Hello
  • You will know:
    • Basic elements of the Pharo/Smalltalk environment including:
      • The Workspace
      • The Transcript
      • The Class Browser
    • How to create new classes in the Pharo environment

What You’ll Need

To install Pharo, start by downloading the zip of latest version from Pharo Downloads. You’ll have the choice between the Cog and Standard VM. Cog is newer and faster, which won’t matter much for our purposes, but why not get the fancier toy when you can?

Unzip “Pharo.app” to the location you want to keep it, “Program Files” will do the job. Once you’ve unzipped it, navigate into the Pharo.app directory and double click the “Pharo” link and your new Pharo system will open.

A Brief Tour of the Environment

Smalltalk was designed to be an entire environment, separate from whatever system that it’s running on. The advantage is it gives you a completely portable programming environment that you can move to any machine that has a Smalltalk VM and it will work the same way on each of them. A disadvantage is that there’s a lot to get used to in it. For the next couple of weeks, 90% of the work is going to be in the System Browser with side trips in a few of the other tools. The major components of it are:

Smalltalk Environment

  • System Browser: This is where you’ll do your actual coding.
    • Packages: Packages are similar to modules, but they only have formal existance when you’re importing them in the system. Once they’re part of your Smalltalk image, their purpose is purely organizational.
    • Classes: Once you’ve selected a package, the classes are the primary element in the system.
    • Protocols: Protocols, like packages, are a way to organize the methods in a class.
    • Methods: Methods are what the classes do.
    • Edit Pane: The Edit Pane is where you’ll do most of your coding including specifying classes and adding methods.
  • Workspace: You can view the Workspace as a scratch pad, you’ll use it when you’re trying to figure something out in the small, or when you’ve completed a piece of work and are doing a bit of ad hoc testing.
  • Transcript: The transcript is where you’ll print logs and quick bits of system interaction. You can think of it as a combination of stderr and stdout in a more conventional system.

Files Used in this Project

  • Virtuous-Demo.st: Contains all of the sample code developed during this project.

To import “Virtuous-Demo.st” into your Smalltalk image:

  1. Open Pharo
  2. Click on the background to open the World menu.
  3. In the World menu select Tools > File Browser
  4. In the File Browser dialog, navigate to the folder that contains “Virtuous-Demo.st”
  5. Select “Virtuous-Demo.st”
  6. Click the “install” button at the top of the File Browser dialog.
  7. Close the File Browser.

You can verify that the package has been correctly installed by doing the following:

  1. In the World menu select System Browser
  2. In the Package pane, scroll to the bottom where you should see a package called “Virtuous-Demo” which contains the class “Greeter”.

To keep it as part of the image (along with any other changes) click “Save” in the World menu or to exit click “Save and quit”.

Executing Code

To execute code in Pharo:

  1. In the World menu, select Workspace, this is where you’ll enter the code to execute.
  2. In the World menu, select “Tools” > “Transcript”, this is where results will be displayed.

In the Workspace window enter the following code:

Transcript show: 'Hello World'; cr.

There are two ways to run the code. You can either right-click on the same line as the code and select “do it” or fellow keyboard addicts can type Alt-d while the cursor is on the same line as the code. In either case the words “Hello World” will then appear in the Transcript followed by a new-line. You might also see some paired numbers that come up during the process, this is a side effect of the Transcript being where logging data for the system in general is sent.

Understanding the code you just Executed

One element at a time:

  • Transcript is the object of interest, generally objects will start with a lowercase letter, but this is a special case
  • show: is a method (message) that is called (sent) to the Transcript object. This says that whatever object it is given should be written to the Transcript window, the : indicates that it takes an argument which is…
  • 'Hello World' is a string, note that Smalltalk strings are surrounded by single, not double quotes.
  • ; indicates the end of one method, but that the following method is also getting sent to the same object.
  • cr another method that is sent to Transcript and writes a new line to the Transcript window.
  • . indicates the end of a statement.

The Code

Coding in Smalltalk

The first step you’ll take in your Smalltalk project is to add it to the Packages pane of the System Browser window. To do this:

  1. Make sure that the Packages pane is selected, left-clicking it will do the trick.
  2. Right-click in the Packages pane and select “add category…”
  3. Enter a name that works for you and click OK. (Mine was “Virtuous-Demo”, remember, this is just to keep your code organized)

Now that you’ve added your category, select it from the Packages pane, notice that in the Edit Pane there’s a template for creating a class:

Object subclass: #NameOfSubclass
    instanceVariableNames: ''
    classVariableNames: ''
    poolDictionaries: ''
    category: 'Virtuous-Demo'

You’ll want to add a new class:

  1. Change ‘#NameOfSubclass’ to ‘#Greeter’
  2. Right click in the Edit pane and select “accept” (or type alt-s) to write your class to the system.

Now you can see the new class in the Classes pane. In the Protocols pane click “as yet unclassified”, this is a generic bucket for any methods you write without sorting them. Now you can see a template for a method in the Edit pane:

messageSelectorAndArgumentNames
    "comment stating purpose of message"

    | temporary variable names |
    statements

Start writing your method, since there’s going to be no need to track temporary data within an object I made this a “class” rather than “instance” method, so click the “class” button underneath the class pane. Now change the Edit pane like so:

sayHello
    "Greet the User"

  Transcript show: 'Hello World'; cr.

The first line is the method name, followed by a comment. I removed the | temporary variable names| because there was no need for them, and put the actual action of the method in the last line.

When you’ve finished entering the method “accept” it the same way you did with the class. When you “accept” a new method for the first time, it will ask you to enter your full name. Smalltalk automatically tracks all changes to the system and it will uses each individual’s name to track who has made the changes.

To try your new method out, type Greeter sayHello. into the Workspace and then execute it.

Arguments in Methods

To get an idea of how arguments, I wrote a version of sayHello that knows who it’s talking to. To add another method click “as yet unclassified” in the Protocols pane again and the Edit pane will be populated with the template again, add the method sayHello: as below:

sayHello: toWho
    "Greet the User"

  Transcript show: ('Hello ', toWho); cr.

Inside the method declaration is the argument list. To indicate that a method has an argument append a : to it and follow it with the argument name. In the body of the method there’s one more bit of Smalltalk syntax the , operator concatenates two strings. Strictly speaking it’s not Smalltalk syntax, but just an operator attached to the String class. I’ll get into the details of operator declaration in a later article.

Try this method by typing Greeter sayHello: 'Professor'.

Final Summary

Nice things about Smalltalk so far:

  • Very simple syntax
  • Very clear code

Difficult things about Smalltalk:

  • Learning the environment
  • Resisting the urge to spend hours just playing with the system instead of working

Coming Up

Next week I’ll be putting together a class for calculating the “FizzBuzz” problem and unit test it using Pharo’s integrated unit test system.

Resources

  • Squeak: Squeak Smalltalk is the basis of many of the modern free Smalltalk implementations
  • Pharo: Pharo is a professional design system based on Squeak
  • Smalltalk Zen: Dimitri Zagidulin’s informative page about web development in Pharo/Squeak
  • Smalltalk Language Notes: A nice reference for a lot of basic Smalltalk programming

Robot Framework

Test Automation with Robot Framework

The Plan

Robot Framework is a functional test framework that makes it possible for its users to:

  • Write tests in easily understood language
  • Organize what tests are run for specific purposes
  • Generate both high level and detailed test results

Notice that what it does not do is any specific testing activity. Instead it acts as a front end for libraries like Selenium and AutoIT which perform the actions themselves. For the purpose of this tutorial I’ll be focusing on web automation using Selenium as the back-end.

What to Expect

When this article is complete:

  • You will have:

    • An installation of Robot Framework with Selenium
    • Several tests for a basic login page
  • You will know:

    • How to use Robot Framework(RF):
      • To drive Selenium
      • To write functional tests
      • To write data-driven tests
      • To select or exclude tests by their tags
    • How to read an RF results file
    • How to read an RF log file
    • To find identifiers on an HTML page

Installing Robot Framework and Selenium

Before installing Robot Framework, you need to install Python 2.5 or later. Robot Framework hasn’t been updated to work with Python 3, so currently 2.7 is your best bet. If you don’t already have Python installed you can download it from here.

Next you’ll need to download Robot Framework and Selenium. For each of the installs select the default values.

Finally, add your python installation and robot installation to your path. If you’ve installed Python 2.7 it will be: “c:\Python27\;c:\Python27\Scripts\;”.

Files Used in this Project

Executing Robot Framework Tests

In Robot Framework each file contains one or more tests. That file is treated as a test suite. Every directory that contains a test suite file or directory is also a test suite. When RF is executed on a directory it will recurse through all files and directories of the correct kind except those that start with an underscore character.

To run the tests in “SimpleTest.txt” navigate to the directory that contains it in the command line and type:

pybot SimpleTest.txt

There are arguments that will give you greater control over what tests are executed, but I’ll get to those later in this article.

Notice that while the test is running there are two windows. One is the window in which the test is being executed, the other is the controller window that Selenium is executing the JavaScript that controls the other window from.

When the test is finished executing four files will have been generated: report.html, log.html, output.xml and selenium_server_log.txt. Of these the only one’s you’ll generally need to be aware of are report.html and log.html.

Reading the Results

Your first stop after a test run is report.html. When you open it, if you’re lucky, you’ll see this happy image:

Passed Report

The background is green when all tests have passed, red if any have failed. If you need more detail than that the “Test Statistics” section gives how many tests have passed and failed sorted by criticality, tag and suite. The “Test Details” section gives how long the test took to run and, if it failed, what the fail message was, as can be seen here:

Failed Report

When a test fails, unless the message makes it immediately obvious, you’ll generally next need to read the log. If you click the failed test name, it will then open up the test logs:

Failed Log

The log gives a detailed view of the execution of each of the tests. While the information for all of the tests are available, the tests and steps that are failed are marked in red and already expanded.

The Code

Syntax and Formats

Robot Framework files are sorted into tables. The format of those tables is flexible, you may create them in tab-separated, html, ReStructured text, or just plain text. I’ve tried writing tests in each of these, HTML makes the prettiest files, and is also a pain to work with. In the end, plain text has been the best combination of nice to read for my customer and easy for me to maintain.

There are up to four tables in a test file, the “Testcases” table is mandatory, it contains all the tests you’re going to run, the optional tables are:

  • Settings: Data needed across tests, like what libraries to use and how to setup and teardown tests.
  • Variables: Variable names and values
  • Keywords: These are function definitions, RF calls them keywords, there’s no difference.

The Test

Given a Login Page verify that when the correct username and password are supplied, it is possible to login.

A First Attempt

Settings

*** Settings ***
Library       Selenium Library

Each Table is marked out by *** <table name> *** and continues until the next table name is reached. The text version of the RF syntax is whitespace delimited. Each item under a table is flush to the left of the screen. If a keyword takes arguments then the follow it on the same line, separated by either a tab or two or more spaces. I generally space them out more than two for readability.

The Library setting imports the named library into the test suite. The Selenium Library allows its users to control a web browser automatically and ideally suited to the task at hand.

Identifying Elements on a Web Page

When automating any system, you have to have a way for your test software to find the elements that you’re interacting with. Selenium does this through one of several methods, depending on how automation friendly the software you’re testing is. If it’s very friendly then every element on the page has either a unique name or id attribute. If it’s more hostile then you can still identify any element through its xpath. I recommend using xpath only as a last resort. It’s a powerful tool, but makes your code hard to understand.

I wrote the login page, so it’s very test friendly. For simpler pages like this, I general just look at the sourcecode of the page to find out how to identify the elements, if you look at the Login Page, you’ll find that the identifiers are:

Login Page

Test Cases

*** Testcases ***
Login Should Succeed When the Correct Username and Password are Entered
  Start Selenium Server
  Open Browser  http://zdiles.chaosnet.org/  ie
  Maximize Browser Window
  Input Text    uname  BUser
  Input Text    pwd    TestPass
  Click Button  login
  Page Should Contain  Welcome
  Close Browser
  Stop Selenium Server

I’ve named this file “SimpleTest.txt”. Here’s a line By line look at it, I’ve informally labeled the below steps with their purpose: Setup, Action, Verification, Teardown

  1. A Testcases table contains one or more tests.
  2. The first line will be the name of the test as used in the reports and logs.
  3. Setup Task: To use Selenium a server has to be started that will receive commands and control a browser. Notice that keyword names can have single spaces in them.
  4. Setup Task: A browser is opened to the page given, by default Firefox is used. If specified as it is here it can also use Internet Explorer or Chrome.
  5. Setup Task: By default the browser window isn’t maximized. This fixes that.
  6. Test Action: The Input Text keyword enters text into a field, it takes two arguments, the identifier for the element and the text to enter. This fills the username field.
  7. Test Action: The next Input Text call fills the password field.
  8. Test Action: One of the nice things about robot framework is that its keyword names tend to be pretty obvious. The argument passed to it identifies the Login Button.
  9. Test Verification: Page Should Contain searches through all the text on the page and passes if it contains the given text and fails otherwise.
  10. Teardown Task: Close Browser, this will shutdown the browser opened in line 4, if not run then it just leaves all the windows opened between tests, each test will ad more leading to clutter and confusion.
  11. Teardown Task: Shuts down the Selenium Server, if this isn’t done then it can occasionally lead to trouble when you try to start the Selenium Server in a future test.

Suite Settings

The test as it’s written will faithfully perform the tests given, but has a couple of problems. When a test fails Robot Framework stops running the test to save time and avoid performing activities on a system that’s in a bad state. This means that if the verification step fails, then our teardown tasks will never be executed. To deal this problem, and make neater more readable test cases there are settings at both the test and suite level to setup and teardown tests.

Setup/Teardown

*** Settings ***
Library         Selenium Library
Suite Setup     Start Selenium Server
Suite Teardown  Stop Selenium Server
Test Setup      Open Browser  http://zdiles.chaosnet.org/  ie 
Test Teardown   Close All Browsers

Setup and Teardown can be set at both the test and the suite level. Each fires off at different points during a test run:

  • Suite Setup: Fires once before any tests during the suite are run.
  • Test Setup: Fires before each test is executed.
  • Test Teardown: Fires at the end of each test execution, even when the test fails.
  • Suite Teardown: Fires when all tests in the suite have completed, even when one of them has failed.

This slims our testcase down considerably:

*** Testcases ***
Login Should Succeed When the Correct Username and Password are Entered
  Maximize Browser Window
  Input Text    uname  BUser
  Input Text    pwd    TestPass
  Click Button  login
  Page Should Contain  Welcome

But we still have the Maximize Browser Window keyword, which really doesn’t improve the reader’s understanding of the intent of the test, and is likely to be used in the execution of any future testcase. Unfortunately the setup/teardown keywords can only take one keyword.

The Keywords Table

You can solve this by adding a keyword that calls both of these:

*** Keywords ***
Setup Test
  Open Browser  http://zdiles.chaosnet.org/  ie
  Maximize Browser Window

And change the Test Setup setting to use it:

*** Settings ***
Library         Selenium Library
Suite Setup     Start Selenium Server
Suite Teardown  Stop Selenium Server
Test Setup      Setup Test
Test Teardown   Close Browser

Making Your Tests More Readable

*** Testcases ***
Login Should Succeed When the Correct Username and Password are Entered
  Input Text    uname  BUser
  Input Text    pwd    TestPass
  Click Button  login
  Page Should Contain  Welcome

Now that I’ve gotten rid of the steps that obscure the intent of my test, it’s time to make the purpose of the tests themselves more clear by creating keywords with more meaningful names than those given, so we’ll add a couple more keywords to our Keywords table:

Enter Username  [Arguments]  ${username}
  Input Text  uname  ${username}

Enter Password  [Arguments]  ${password}
  Input Text  pwd  ${password}

Click the Login Button
  Click Button  login

Login Is Successful
  Page Should Contain  Welcome

Which introduces the use of settings in individual keywords. The syntax used for keyword and test settings(to be seen soon) is the same. A special element surrounded in square brackets, you can find a full listing of these in Robot Framework’s User Guide. Here I’m able to specify arguments that will be passed into the keywords from the test case that calls it. The syntax for variables is ${<variable name>} and can contain spaces.

Now my testcase is considerably more readable:

Login Should Succeed When the Correct Username and Password are Entered
  Enter Username  AUser
  Enter Password  TestPass
  Click the Login Button
  Login Is Successful

The Power of Tags

While it’s useful to be able to run every test in your test suite at once, often you only need to run a small subset of them to perform a smoke test, or to test only a specific feature. To this end you can tag tests and suites to indicate their function.

The code below is the same as what I’ve constructed with one more test, which should fail every time, and a couple of tags (Keywords left out because they don’t change):

*** Settings ***
Library         Selenium Library
Resource        LoginKeywords.txt
Suite Setup     Start Selenium Server
Suite Teardown  Stop Selenium Server
Test Setup      Login Test Setup
Test Teardown   Login Test Teardown
Force Tags      FunctionalTest
Default Tags    ValidTest

*** Testcases ***
Login Should Succeed When No Username Is Given
  [Tags]  InvalidTest
  Enter Username  ${empty}
  Enter Password  TestPass
  Click the Login Button
  Login Is Successful

Login Should Succeed When the Correct Username and Password are Entered
  Enter Username  AUser
  Enter Password  TestPass
  Click the Login Button
  Login Is Successful

One Test level setting has been added, tagging the first test with InvalidTest. At the suite level there are two more. Force Tags ensures that every test in the suite will be tagged as a FunctionalTest. Default Tags set’s a test’s tags to ValidTest unless the test has a tag setting of its own like the first test.

There are two benefits from adding descriptive tags to your tests. When you run your tests your test results will be sorted by them. This will frequently help determine where an individual problem is. Additionally you can determine which tests get executed. For example when you use:

pybot -i InvalidTest SimpleTest.txt

Will only include the tests with that tag, while:

pybot -e InvalidTest SimpleTest.txt

Will exclude any test with that tag.

Keyword Files

Keywords like Enter Username will be used in nearly every test you write for this page. It’s often helpful to break up your test suites into multiple files, or even multiple directories. To avoid copy pasting the same set of keywords into every file, you can break them out into a separate resource file. The easiest way in our current case is to just cut everything from *** Keywords *** on down and put it in a new file called “LoginKeywords.txt”

We’re getting to a point where this is well beyond a simple test, so while you’re at it rename “SimpleTest.txt” to “LoginTest.txt” and add one more line to the Settings section of LoginTest.txt, like so:

*** Settings ***
Library         Selenium Library
Resource        LoginKeywords.txt
Suite Setup     Start Selenium Server
Suite Teardown  Stop Selenium Server
Test Setup      Login Test Setup
Test Teardown   Login Test Teardown
Force Tags      FunctionalTest
Default Tags    ValidTest

Directories as Suites

I’m going to add more test files, so to organize them I’m going to create a directory “LoginTest” and put “LoginTest.txt” and “LoginKeywords.txt” in it. Now if I execute pybot from the directory above the “LoginTest” directory as:

pybot LoginTest

It will execute all of the test files in LoginTest.

Init Files

As stated above, both directories and files are treated as test suites by Robot Framework. You can create a Settings section for a directory by adding the file “__init__.txt” and putting a Settings table in it. Since I’m about to add more tests and they’re all going to need the same setup and teardowns, I’ll move a few more settings from “LoginTest.txt” to “__init__.txt” giving us:

*** Settings ***
Library         Selenium Library
Resource        LoginKeywords.txt
Suite Setup     Start Selenium Server
Suite Teardown  Stop Selenium Server
Test Setup      Login Test Setup
Test Teardown   Login Test Teardown

And now the Settings section of “LoginTest.txt” is:

*** Settings ***
Library         Selenium Library
Resource        LoginKeywords.txt
Force Tags      FunctionalTest
Default Tags    ValidTest

Data Driven Tests

Data driven tests are designed to repeatedly run the same test with changes in the data is being used for inputs. This allows the person writing the tests to create a large number of tests scenarios with minimal effort.

Robot Framework has a special setting called Test Template that lets you create files that are dedicated to Data Driven test suites:

*** Settings ***
Library         Selenium Library
Resource        LoginKeywords.txt
Test Template   Login Should Fail When
Force Tags      DataDriven  ValidTest

*** Testcases ***       Username  Password
The Password is Wrong   AUser     TestFail
The Password is Empty   AUser     ${empty} 
Both Fields are Empty   ${empty}  ${empty} 

*** Keywords ***
Login Should Fail When   [Arguments]  ${username}  ${password}
  Enter Username  ${username}
  Enter Password  ${password}
  Click the Login Button
  Login Is Unsuccessful

Using Login Should Fail When as the template means that each of the three testcases use it as their only keyword, and apply the arguments given to it. You can see on the first line of the Testcases table there are two columns set “Username” and “Password”. Robot Framework ignores anything in the first line of the table after the name, the column names are strictly for the reader’s benefit. The other new element is the ${empty} variable, this indicates an empty string.

For this suite you can see I’ve added one more new keyword to “LoginKeywords.txt” called “Login Is Unsuccessful.” It is the negation of “Login Is Successful”:

Login is Unsuccessful
  Run Keyword And Expect Error  *  Login Is Successful

Run Keyword and Expect Error does exactly what it says. The second argument is the error expected, * will allow for any error. The third argument is the keyword to run, then the follow arguments get passed to the keyword in the third argument. This is a useful way to define negative test cases.

Variables

When your tests get even moderately complicated there’s going to be data that needs to change from one run to the next or depending on the context you’re running it in. Use a variable table to minimize the pain of dealing with this. For my tests I have two variables that I’m concerned with, the URL under test and the browser used, pulling these out as variables in the keyword file gives me:

*** Variables ***
${login address}=  http://zdiles.chaosnet.org/
${browser}=  ie


*** Keywords ***
Enter Username  [Arguments]  ${username}
  Input Text  uname  ${username}

Enter Password  [Arguments]  ${password}
  Input Text  pwd  ${password}

Click the Login Button
  Click Button  login
 
Login Is Successful
  Page Should Contain  Welcome

Login is Unsuccessful
  Run Keyword And Expect Error  *  Login Is Successful

Login Test Setup
  Open Browser  ${login address}  ${browser}
  Maximize Browser Window 

Notice that when they’re assigned the variable names have an = appended to them, this is optional, but it makes the code a little more clear.

Final Summary

I’ve found Robot Framework to be powerful tool for designing and executing tests. By adding functional tests to your project you can improve its overall quality. Enjoy.

Coming Up

Next week I’ll start with Smalltalk, using the Pharo environment which is redesigned from Squeak to be a powerful professional development tool.

Resources

Prolog 4/4: GUI

Prolog 4/4: GUI

The Plan

I will write a GUI for last week’s RSS reader in order to explore how to write a GUI in a logic based language using Prolog’s XPCE toolkit.

The XPCE toolkit is:

  • An IDE for SWI-Prolog.
  • A library for writing GUI in Prolog.
  • An object layer for Prolog.

What to Expect

When this article is complete:

  • You will have:
    • A GUI front end for the RSS reader from last week
  • You will know:
    • How to use SWIPL-Win.
    • How to write a GUI in Prolog.

Files Used in this Project

  • rssGui.txt: The GUI code. (Change the extension from txt to pl)
  • rss.txt: The original RSS reader file, used here as a library. (Change the extension from txt to pl)
  • feeds.txt: Sample RSS feed config file.

The Code

Compiling the Code… More or Less

I discovered that compiling GUI code for XPCE is easier said than done. I made several attempts, read a couple of tutorials and got nowhere. If there are any Prolog guru’s reading who wouldd like to set me straight, please leave a comment.

On the other hand, running the code on the is pretty easy:

  1. Download the files for this project and re-name them as described above.
  2. Double click on “rssGui.pl”
  3. In the SWIPL window, type, rssGui.

At that point, you should have a window with the RSS reader in it.

Loading Libraries And Other Activities

:- consult(rss).
:- pce_autoload(finder, library(find_file)).
:- pce_global(@finder, new(finder)).

To load the rss library from last week, tell Prolog to consult it. The two lines afterward load the file finder into memory. This makes it possible to open a file finder to select the rss config file.

Laying Out the GUI, and Adding Hooks

rssGui :-
  new(Frame, frame('RSS Reader')),
  new(RssFeeds, browser),
  send(Frame, append(RssFeeds)),
  send(new(NewsList, browser(news)), right, RssFeeds),
  send(RssFeeds, select_message, message(@prolog, newsDisplay, RssFeeds, NewsList)),
  send(new(Buttons, dialog), below(RssFeeds)),
  send(Buttons, append(button(load, message(@prolog, load, RssFeeds)))),
  send(Frame, open).

XPCE makes it possible to access objects in Prolog by adding three predicates:

  • new: Instantiates a new object.
  • send: Sends information to the object.
  • get: Gets information from an object.

To make a new GUI window, first a frame is created(Line 2) which contains all the GUI elements.
Lines 3 and 4 create a List Box to contain the channel names and adds it to the Frame.
Line 5 adds a List Box to the right side of the channel list box for displaying news titles.
Line 6 adds an action so that when a value is selected in RssFeeds, the newsDisplay predicate is fired with RssFeeds and NewsList for arguments.
Line 7 adds a container for a button. Line 8 adds a button to the dialog which has the text “load” and fires load when clicked.
Line 9 opens the frame.

Read the Config and Load the RSS Feeds

load(Browser) :-
  get(@finder, file, exists := @on, FileName),
  readConfig(FileName, URLs),
  send(Browser, clear),
  foreach(member(URL, URLs), readAndListChannels(Browser, URL)).

Line 2 opens up a standard file dialog and unifies the selected file with FileName, from there the code is much like the display code from last week.

I used a separate predicate to write each of the URLs to avoid potential logical contradictions in the foreach.

Fetch the Feed and Add it to the RssFeeds Browser

readAndListChannels(Browser, URL) :-
  catch((rssFetch(URL, XML),
        rssRead(XML, RSS),
        foreach(member(channel(Name, News), RSS), 
          send(Browser, append, create(dict_item, Name, Name, News)))), 
      _, 
      readAndListChannels(Browser, URL)).

Again, much of this is similar to the code to display the feeds on the command line. The only real change is in Line 5, where the data from each channel is wrapped into a dictionary item with:

  1. identifier: Name
  2. key value: Name
  3. object: News

As before, this will keep trying until it succeeds or blows the stack.

Display the News Titles in the NewsList Browser

newsDisplay(RssFeeds, NewsList) :-
  get(RssFeeds, selection, Channel),
  get(Channel, object, News), 
  get(News, size, SizeVal),
  send(NewsList, clear),
  foreach(between(1, SizeVal, LineNumber), 
      sendVector(NewsList, News, LineNumber)).

Line 2 unifies Channel with the currently selected item in RssFeeds, the dictionary item from readAndListChannels.
Line 3 pulls the object from the dictionary item, a vector.
Line 4 determines the length of the vector.
Line 5 clears the NewsList so we don’t just keep adding to the end.
Line 6-7 iterates over each of the values in the vector and appends them to NewsList with sendVector.

Append a Cell from a Vector to a Browser

sendVector(Browser, Vector, Index) :-
  get(Vector, element, Index, Value),
  send(Browser, append, Value).

Given a Browser, Vector and Index, sendVector pulls the Indexth element from the Vector and appends it to the Browser.

Final Summary

Prolog’s logic based semantics make for an interesting programming experience. Up to this point I had trouble imagining how you’d deal with dynamic situations, like user input or a file being read in, in context to a language that is based on logic. Now that I’ve spent a month working with it, it’s an incredibly elegant way to solve problems. That being said, I think I’m going to stick with Python for my day to day scripting needs.

Coming Up

Next week is a fifth Monday, so I’ll be working with a tool, Robot Framework. Robot Framework is a Python based glue language for writing test cases in a clear format which is readable to non-technical users.

In February, I’ll be working with Squeak. Squeak several neat features, it is:

  • generally considered the first full object oriented language
  • where unit tests frameworks came from (in the form that most of us are used to today)
  • contained entirely in its own design environment
  • designed to be useful as an education language
  • host to Seaside a powerful web framework

Resources

Prolog 3/4: XML Parsing

Reading, Parsing and Displaying RSS Data in Prolog

This is the third article in a series, to read it from the beginning go here.

The Plan

This week is when I really put the assertion that Prolog is a general purpose language to the test. Most decently rational Prolog programmers will tell you to do things like XML parsing and display through another language, then use Prolog dll’s to do the interesting logical stuff. But, it does have the libraries, so let’s see how they work.

What to Expect

When this article is complete:

  • You will have:
    • A command-line RSS aggregator.
  • You will know:
    • How to read a text file.
    • How to read from an HTTP stream.
    • How to parse an XML file.
    • How to pretty print text to the screen.

Files Used in this Project

Compiling

To compile “xml.pl”, navigate to the directory that contains it on the commandline, then execute:

swipl -g main -o xml.exe -c xml.pl

The Code

Libraries Used

:- consult(library(sgml)).
:- consult(library(http/http_open)).
  • sgml: The SGML library, which also handles XML.
  • http/http_open: The HTTP library handles server and client code, I just used the small bit that let’s me treat an HTTP stream like a file.

Reading a File

readConfig(Lines) :-
  open('feeds.txt', read, ConfigStream),
  read_stream(ConfigStream, Lines).

read_stream(ReadStream, Lines) :-
  read_line_to_codes(ReadStream, Line),
  read_stream(ReadStream, Line, Lines).
read_stream(ReadStream, end_of_file, []) :- close(ReadStream), !.
read_stream(ReadStream, OldLine, [Atom | Lines]) :-
  read_line_to_codes(ReadStream, NewLine),
  string_to_atom(OldLine, Atom),
  read_stream(ReadStream, NewLine, Lines).

read_stream is a helper function to read multiple lines from a stream. It reads a stream in one line a time and returns/unifies its contents to Lines. Most of it is similar to what you saw in the previous article, recursively pulling off one line at a time, testing if it’s the last line and adding it to the list. The two key elements that are worth noting are: the use of the atom end_of_file, which is what the file stream returns when the stream is empty and the use of ! to indicate that if execution makes it to closing the stream, then there’s no need to backtrack.

readConfig‘s functionality is pretty obvious. Open a file using an atom with the file’s name. Remember, single-quotes are for atoms, double quotes are for strings. This opens a read only stream and unifies it with ConfigStream, then read_stream pulls the text in the file out into Lines.

Reading From HTTP

rssFetch(URL, XML) :-
  http_open(URL, XmlStream, []),
  load_xml_file(XmlStream, XML),
  close(XmlStream).

The pattern here is similar to the one for readConfig, except that it extracts the XmlStream straight to a Prolog XML structure. There’s quite a bit to it, but the short version is, it’s a nested linked list. There are a variety of functors in that list, but the one that interests us is element which has the form element(<element name>, <attributes>, <sub elements>). For example when given the xml:


  in text

load_xml_file returns:

[element(testOuter, [], 
    ['\n  ', element(inside, [inAtt=hello], ['in text']), '\n'])].

Parsing XML

rssRead([], []).
rssRead([element(channel, _, Elements) | _], [Rss | Rsses]) :-
  channelRead(Elements, Rss), rssRead(Elements, Rsses).
rssRead([element(rss, _, Elements) | _], Rss) :-
  rssRead(Elements, Rss).
rssRead([_ | Elements], Rss) :-
  rssRead(Elements, Rss).

channelRead(Elements, channel(Name, Titles)) :-
  titleRead(Elements, Name), itemsRead(Elements, Titles).

itemsRead([], []).
itemsRead([element(item, _, Elements) | Items], [ Title | Titles ]) :-
  titleRead(Elements, Title), itemsRead(Items, Titles).
itemsRead([_ | Items], Titles) :-
  itemsRead(Items, Titles).

titleRead([element(title, _, [Text | _]) | _], Text) :- !.
titleRead([_ | Elements], Text) :-
  titleRead(Elements, Text).

Strictly speaking Prolog is a weakly typed language, you have only a couple of basic types which all reduce to atoms and functors. But with those you can do the same tricks that are generally associated to algebraic type systems. Here is one example where I use the element functor to recurse through the XML structure and extract the titles.

Once you know how Prolog’s XML structures work, it’s just a matter of figuring out what information you need and recursing until you get it. For this purpose I’ve created my own functor channel(<channel name>, <title list>).

In Prolog functors and atoms are created simply by using them. This makes writing the code feel a lot like freeform sketching a picture. I like it. But I’m pretty sure that unless I made an effort to document my code well, it would make large projects unwieldy.

Pretty Printing Text

displayRss(Channels) :-
  foreach(member(channel(Name, Titles), Channels),
      (writef("*** %w ***\n", [Name]), displayTitles(Titles), nl)).

displayTitles(Titles) :-
  foreach(member(Title, Titles), writef("\t%w\n", [Title])).

writef works similarly to printf in nearly every language I’ve worked in. It has a few quirks that are detailed in its docs.

Main And Helper Text

readAndDisplayRss(URL) :-
  catch((rssFetch(URL, XML), 
        rssRead(XML, RSS), 
        displayRss(RSS)), _, readAndDisplayRss(URL)).

main :-
  readConfig(FeedURLs),
  foreach(member(URL, FeedURLs), readAndDisplayRss(URL)),
  halt.  

The above is glue code. It’s been written to continue retrying until it succeeds or runs out of stack, so make sure your URL’s are valid.

Final Summary

When I was reading up on Prolog, I heard predicates described as a generalization on functions. This week has driven that one home. Working in Prolog has felt a great deal like when I was first learning Haskell. I’ll spend hours fighting with something that I could do in my sleep in another language, then when I figure it out it is extraordinarily clear, obvious and feels cleaner than anything else I’ve worked with.

Hard Lessons Learned

foreach is not the same as a for comprehension/list comprehension, no matter how much it looks like one. The key difference is that the second term in the statement does not contain its own namespace. What happens in there happens throughout the same namespace that contains the foreach. For example:

main :-
  foreach(member(Number, [1, 2, 3, 4]), 
      (Square is Number * Number,
       write(Square), nl)).

Does not print the square of the numbers 1 through 4. It first asserts that Square is equal to 1 * 1, then that it is also equal to 2 * 2, 3 * 3, and 4 * 4, which is clearly false and ends execution of the foreach. It is possible to do this, the safe and idiomatic way is to break the second term out into its own predicate:

showSquare(Number) :-
  Square is Number * Number,
  write(Square), nl.

main :-
  foreach(member(Number, [1, 2, 3, 4]), showSquare(Number)).

That lesson took me half a day for me to learn. You’re welcome.

Coming Up

Next week I’ll be working out how to use XPCE, Prolog’s native GUI library.

Resources

Prolog 2/4: Loops, Decisions and Tests

Writing and Testing Fizzbuzz in Prolog

This is the second article in a series, to read it from the beginning go here.

The Plan

This week I’ve worked my way through solving the Fizzbuzz problem in Prolog. The most difficult part of working on it was learning to deal with Prolog’s syntax. Once I got a decent grasp of the syntax, the solution to the fizzbuzz problem fell out quite naturally.

What to Expect

When this article is complete:

  • You will have:
    • An executable that produces fizzbuzz up to 20
    • Unit tests that verify several of predicates
  • You will know:
    • How to write recursive clauses.
    • How to write conditionals.
    • How to write loops.
    • How to trow and catch exceptions.
    • How to write and run unit tests.

Files Used in this Project

The Code

Basic Predicates

isFizz(Num) :- 0 is Num mod 3.
isBuzz(Num) :- 0 is Num mod 5.
isFizzbuzz(Num) :- isFizz(Num), isBuzz(Num).

There are two basic ways that information comes out of a predicate. The first is by filling in all of its arguments, then it returns either true or false depending on how they evaluate in the predicate’s body.

Conditionals

fizzbuzz(Num, Res) :- 
  isFizzbuzz(Num) -> Res = 'fizzbuzz';
    isFizz(Num) -> Res = 'fizz';
    isBuzz(Num) -> Res = 'buzz';
    Res = Num.

Prolog’s conditional is ->. a -> b is equivalent to if a then b. You can see above that the conditionals are separated out by Prolog’s “or” operator is ;. In this context, ; operates as an else. These aren’t special cases, but the standard operation of Prolog. For example:

  1. If isFizzbuzz(Num) evaluates to true, then Res = ‘fizzbuzz’ and the statement evaluates to true and the following ‘or’ statement is short-circuited.
  2. If isFizzbuzz(Num) evaluates to false, then the whole if statement evaluates to false and then the other side of the ‘or’ operator is evaluated.

Recursion and Exceptions

fizzbuzzes(TopNum, TopNum, List) :-
  List = [],!.
fizzbuzzes(TopNum, CurrentNum, [Head | Tail]) :- 
  CurrentNum > TopNum -> throw('the CurrentNum is greater than TopNum'); 
  TopNum < 1 -> throw('the TopNum is less than 1'); 
  (NextNum is CurrentNum + 1,
  fizzbuzz(CurrentNum, Head),
  fizzbuzzes(TopNum, NextNum, Tail)).

Here I took advantage of the fact that Prolog evaluates its terms on the left side as well as on the right of :-. For the first clause, if the first two terms have the same value then this clause is the one to fire. List is set to [], then all further evaluation stops because of the !. I’m still a little vague on exactly how it works, but the short version is that if a predicate has multiple possible values, then Prolog will normally try to look for them. The ! tells Prolog that once it reaches it, it shouldn’t look any further.

Line 3 is in some ways the oddest one in the set. The first two values passed in aren’t unusual. The third value [Head | Tail] is a list which doesn’t have a name as a whole, but its head and tail do. For our use this is will be the return value of the predicate, though in Prolog this isn’t entirely a meaningful phrase. Later you’ll see a case where the “return value” is passed into the predicate to verify if it’s valid.

In Line 4 I verify if the CurrentNum is greater than the TopNum, this should never happen since there’s a wrapper that’s intended to be used that won’t let it, but there’s no harm in a bit of verification. If it is then an error is thrown with the term: 'the CurrentNum is greater than TopNum'. Normally terms don’t have spaces in them, but if you surround a term in single quotes it can have any combination of characters.

Clauses with More Than One Arity

fizzbuzzes(TopNum, List) :- 
  OneHigher is TopNum + 1,
  fizzbuzzes(OneHigher, 1, List).

This fizzbuzzes is the same clause as the one in the previous section, but with Arity 2 instead of 3. In Prolog these are differentiated as fizzbuzzes\2 and fizzbuzzes\3. In the second line you can see some arithmetic, which is OneHigher = TopNum + 1.

Loops

printFizzbuzzes(TopNum) :-
  fizzbuzzes(TopNum, FizzbuzzList),
  forall(member(X, FizzbuzzList), (print(X), nl)). 

main :- 
  printFizzbuzzes(20), 
  halt.

I claimed in the last article that Prolog had no looping constructs. This is a half truth. It has no special syntax for loops, but it does allow for the creation of a for loop as a predicate. forall is part of swipl’s standard library where the first argument is a generator for terms and the second argument is applied to each of those terms.

Unit Tests

Starting Unit Tests

:- begin_tests(fizzbuzz).

A clause that has a body, and not a head is a message to the compiler. A sequence of unit tests always starts with the being_tests clause. The term inside it will mark out what test suite this is.

Test Cases, Negation

test(isFizz_isnot) :-
  not(isFizz(5)).

When a predicate named test is defined it will be added to the test suite. not here is a little bit strange, it does not mean that something is false, but that it can’t be asserted as true. So if there is no assertion of the truth or falsehood of a statement, it will return true.

Not Really a Return Type

test(fizzbuzz_fizz) :-
  fizzbuzz(3, 'fizz').

test(fizzbuzzes_empty) :-
  fizzbuzzes(0, []).

test(fizzbuzzes_5) :-
  fizzbuzzes(5, [1, 2, fizz, 4, buzz]).

Where previously I used the second argument in fizzbuzz and fizzbuzzes to return a value, here I’m using it to verify the truth of a statement.

Closing the Test Suite

:- end_tests(fizzbuzz).

end_tests closes out the test suite.

Running Your Tests

To execute the unit tests, run swipl from the directory that contains fizzbuzz.pl. Load fizzbuzz into the interpreter with [fizzbuzz]., then run the tests with run_tests.

Final Summary

Fizzbuzz comes out well in Prolog. I’ve spent a lot more time time thinking I was fighting with the syntax when the reality was I don’t get the semantics. It’s the first time in a few years I’ve had to remind myself, “The compiler’s not broken, you just don’t know what you’re doing.”

Coming Up

Next week we’ll put the rubber to the road. Prolog’s pretty cool for algoritmic stuff, let’s see how it handles networking and XML.

Resources

Prolog 1/4: Getting Started

Getting Started with Prolog

The Plan

Prolog is rather unusual when compared to other programming languages. Rather than describing a sequence of steps as in a procedural language, it’s a declarative language that uses formal logic as its primary metaphor. Prolog is used predominantly in academic settings for language research and artificial intelligence. That being said it is a general purpose programming language and used in commercial settings where the system must be able to reason about complex data like logistics, language processing and data mining. Some examples can be found at SICStus’s Website.

Although it’s a general purpose programming language, generally other languages are used to do GUI and interface coding. There are GUI packages available though, so I’ll give one of them a try in Week 4. With a little luck we’ll see Prolog’s strength in Weeks 2 and 3.

What to Expect

When this article is complete:

  • You will have:
    • An installation of SWI-Prolog
    • A program that’ll say “Hello World”
  • You will know:
    • How to compile a program in SWI-Prolog.
    • How to use the interactive interpreter to make logical queries from a logic base.
    • That Socrates is mortal.

Files Used in this Project

SWI-Prolog

There are a large variety of implementations of Prolog. From these I chose SWI-Prolog because:

  • It works on Windows. (All projects on this blog do)
  • It has a GUI toolkit. (necessary for week 4)
  • It’s free, under the LGPL. (Please remember to include a link to SWI’s page if you use it for your applications)
  • It has an interactive interpreter. (I like interactive interpreters)

Installation

  1. Download the latest version of SWI-Prolog from their download page.
  2. Run the installer accepting all the defaults.
  3. Add "C:\Program Files\pl\bin" to your PATH in environment variables. How to change your path.

The Code

helloworld.txt (Rename to helloworld.pl):

% Say Hello Prolog
main :-
  write('Hello World!'), nl,
  halt.

Comments

In Prolog comments are preceded by a %.

Clause Structure

Prolog programs are composed of clauses separated by periods. A clause is divided into the head and the body with the format <head> :- <body> or just <head>. When just the head is given it’s treated like <head> :- true. The body of a clause is a statement composed of either a term, or sequence of terms composed by operators and ending in a period.

In Prolog a , is equivalent to boolean and. In the above, should any of the statements return false, then the others will not be executed. Each of the terms: * write('Hello World!'): Writes “Hello World!” to the screen. * nl: Writes a new line to the screen. * halt: Causes Prolog to shutdown.

Running/Compiling Your Code

SWI-Prolog has an interactive interpreter, an interpreter and a compiler.

The Interactive Interpreter

A session with the interactive interpreter looks like:

C:\>swipl
Welcome to SWI-Prolog (Multi-threaded, 32 bits, Version 5.10.2)
Copyright (c) 1990-2010 University of Amsterdam, VU Amsterdam
SWI-Prolog comes with ABSOLUTELY NO WARRANTY. This is free software,
and you are welcome to redistribute it under certain conditions.
Please visit http://www.swi-prolog.org for details.

For help, use ?- help(Topic). or ?- apropos(Word).

1 ?- write('Hello World!'), nl.
Hello World!
true.

2 ?- halt.

C:\Documents and Settings\fberthold\My Documents\My Dropbox\VirtuousProgrammer\Prolog\1 - Getting Started>

You can also interact with the interpreter through SWI-Prolog’s IDE. I’m a command line junky so haven’t played with it too much yet, but you can find it in the Start menu.

The Interpreter

To run your programmer through the interpreter type:

swipl -g main helloworld.pl

Where “helloworld.pl” is your Prolog program, commonly called a logicbase. “main” is what you’ve declared to be the main clause in your program.

The Compiler

You can compile it with:

swipl -g main -o helloworld.exe -c helloworld.pl

This will generate a windows executable “helloworld.exe” by compiling “helloworld.pl”.

Prolog’s Hello World

It’s a little hard to do Prolog justice from this first example, because the primary function of Prolog is to describe and evaluate relationships, which don’t come up in ‘hello world’. To give a little more flavor of what Prolog does, here’s a simple relationship:

socrates.txt (Rename to socrates.pl):

% All men are mortal, Socrates is a man.
mortal(X) :- man(X).
man(socrates).

Here we have two assertions. That men are mortal and that Socrates is a man. We can how use Prolog to draw a couple of conclusions. The first step is to load “socrates.pl” into the interactive interpreter with:

swipl socrates.pl

Here’s a sample session in which we can find that Socrates is mortal and that if you are mortal, you are Socrates (Add more facts to the logicbase if you want more mortals):

1 ?- mortal(socrates).
true.

2 ?- mortal(X).
X = socrates.

Prolog’s single data type is a term. Prolog terms can be(with examples from the above code):

  • atoms: socrates
  • numbers: no example in the above code
  • variables: X
  • compound terms: man(socrates)
    • where man is called a functor

Summary

Prolog is unlike any programming language I’ve worked with to date. Not only is it’s model entirely different, but it’s syntax was designed before it was obvious Algol style syntax was going to predominate, so it’s syntax is more influenced by it’s logical roots than what is currently considered normal looking syntax.

Coming Up

Next week I’ll be putting Prolog through it’s paces. The fizzbuzz problem should be interesting to solve. It feels like it should naturally be able to deal with conditional statements, but it doesn’t have any direct looping facilities other than recursion. Unit tests on the other hand also feel like a natural fit.

Resources

Trivia

  • Despite being based on an entirely different semantic set than other programming languages, Prolog is Turing complete.
  • Prolog was created at the University of Aix-Marseille by Alain Colmerauer and Phillipe Roussel, collaborating with Robert Kowalski of the University of Edinburgh in 1972.
  • If you enter X. at the interactive interpreter, SWI-Prolog gives Douglas Adams fans the answer.

Scala 4/4: GUI

Writing GUI Code in Scala

This is the last article in a series, to read the whole series go here.

The Plan

For the last week of Scala I’ve put together a GUI for week 3’s RSS aggregator using the Swing library. Scala’s Swing library is a fairly thin wrapper around Java’s Swing library. The primary difference between them is that Scala’s library takes advantage of its more powerful type system.

What to Expect

When this article is complete:

  • You will have:
    • A front end for the RSS agregator I wrote in week 3.
  • You will know:
    • How to write a basic Swing application in Scala.
Screenshot:

RSS GUI

Files Used in this Project

The Code

Libraries

import RssReader._
import swing._

You can use the libraries that you’ve written in the same way you call any other Java class library. Remember that you must compile RssReader.scala before you try to compile RssGui.scala.

A Top Level Object

object RssGui extends SimpleSwingApplication {

The SimpleSwingApplication incorporates all of the basics that you need for a standard windowed application in order to: create, draw and destroy the window when it’s closed.

Container for RSS Data

  var rssFeeds = List(("None Loaded", Seq("")))

This will hold the data for the RSS feeds. I’ve chosen to initialize it with "None Loaded" and an empty string because it gives the GUI something to display until a config file is selected and loaded, and it let’s Scala know what the rssFeeds‘s type is.

File Chooser

  var configFileChooser = new FileChooser(new java.io.File("./"))
  var configFileName = new String

The configFileChooser is a FileChooser that will open, defaulting to the current working directory. configFileName will hold the value selected, the details for how this will work are in the “Defining Behaviors” section.

Layout

For the most part the code here stands for itself, which is wonderful. Once you understand the basics for creation and layout for one component, they’re all the same. As a result I’ve be described each new idea in detail when I encountered it and let the rest alone.

Channel Panel Construction
Channel Select
  val channelLabel = new Label {
    text = "Select a channel:"
  }

  val channelSelect = new ComboBox(rssFeeds(0)._1) { }

  val channelSelectPanel = new BoxPanel(Orientation.Vertical) {
    contents += channelLabel
    contents += channelSelect
    border = Swing.EmptyBorder(0, 0, 0, 30)
  }

Lines 1-3: Create a new label with “Select a channel:” for its value.
Line 5: Create a ComboBox that is initialized to the first tuple value in the first list value: “None Loaded”
Line 7: Create a new panel that is filled top to bottom.
Line 8-9: Add the Label and ComboBox to the panel.
Line 10: Surround the panel in an empty border with 30 pixels on the right side (to separate it from the RssDisplay).

Rss Display
  val rssDisplay = new ListView(rssFeeds(0)._2)

This creates a nearly empty ListView. It has one line with an empty string. Until it’s populated with something more complex, it won’t display at all.

Combining channelPanel
  val channelPanel = new BoxPanel(Orientation.Horizontal) {
    contents += channelSelectPanel
    contents += rssDisplay
    border = Swing.EmptyBorder(0, 0, 30, 0)
  }

This creates a new panel with Orientation.Horizontal so each element will be added left to right. The channelSelectPanel and rssDisplay are added to complete the channelPanel.

Button Panel Construction
  val setConfigFileButton = new Button {
    text = "Set Config File"
  }

  val loadRssButton = new Button {
    text = "Load Feeds"
  }

  val buttonPanel = new BoxPanel(Orientation.Horizontal) {
    contents += setConfigFileButton
    contents += loadRssButton
  }

Create the two buttons and lays them out in a panel.

Top Window
  val topWindow = new BoxPanel(Orientation.Vertical) {
    contents += channelPanel
    contents += buttonPanel
    border = Swing.EmptyBorder(10, 10, 10, 10)
  }

Add the two top level panels together and put a border around them to make it neat.

Defining Behaviors

The Last Bit of Layout

  def top = new MainFrame {
    title = "Rss Reader"

    // Overall construction
    contents = topWindow

This creates the overall window and assigns topWindow as its contents. The title “Rss Reader” will appear in the title bar of the window.

Listening

    listenTo(channelSelect.selection, setConfigFileButton, loadRssButton)

Each of the GUI elements that you want to be able to have act as controllers must be listed in listenTo. I’ve chosen to add them all at once, you can also add them one at a time as in:

   listenTo(channelSelect.selection)
   listenTo(setConfigFileButton)
   listenTo(loadRssButton)

To determine when a value has been selected we are listening to channelSelect.selection instead of channelSelect.

Reacting

For each action performed on one of the listenTo‘ed elements it will run through each of the listed reactions to see if any of them are appropriate for the action/object combination.

    reactions += {
      case swing.event.SelectionChanged(`channelSelect`) =>
        rssDisplay.listData = rssFeeds(channelSelect.selection.index)._2
        pack
      case swing.event.ButtonClicked(`setConfigFileButton`) =>
        if(configFileChooser.showOpenDialog(channelSelectPanel) ==
            FileChooser.Result.Approve) {
          configFileName = configFileChooser.selectedFile.getAbsolutePath
        }
      case swing.event.ButtonClicked(`loadRssButton`) =>
        if(configFileName != "") {
          rssFeeds = combineFeeds(getRssFeeds(configFileName))
          channelSelect.peer.setModel(ComboBox.newConstantModel(rssFeeds.map(_._1)))
          rssDisplay.listData = rssFeeds(0)._2
          pack
        }
    }
  }
}

For readability I’ve left the entire set intact above, I’ll look at them for each case here.

A New Channel is Selected
      case swing.event.SelectionChanged(\`channelSelect\`) =>
        rssDisplay.listData = rssFeeds(channelSelect.selection.index)._2
        pack

Line 1: The case looks for the event, SelectionChanged and to which element that event is applied, channelSelect, the backquotes above are important.
Line 2: When the selection changes, set the display to contain the story titles from the rssFeeds value with the same index as the value selected.
Line 3: Once the values have been changed, packs the window so everything fits.

The Config File Button is Clicked
      case swing.event.ButtonClicked(\`setConfigFileButton\`) =>
        if(configFileChooser.showOpenDialog(channelSelectPanel) ==
            FileChooser.Result.Approve) {
          configFileName = configFileChooser.selectedFile.getAbsolutePath
        }

Line 2-3: Opens the configFileChooser and checks to make sure that a value has come back.
Line 4: Assigns the absolute path to the selected file to configFileName.

The Load Rss Button is Clicked
      case swing.event.ButtonClicked(\`loadRssButton\`) =>
        if(configFileName != "") {
          rssFeeds = combineFeeds(getRssFeeds(configFileName))
          channelSelect.peer.setModel(ComboBox.newConstantModel(rssFeeds.map(_._1)))
          rssDisplay.listData = rssFeeds(0)._2
          pack
        }

Line 2: Verifies that some value has been assigned to configFileName.
Line 3: Uses combineFeeds from RssReader to read in the RSS values.
Line 4: Set’s the channelSelect ComboBox to contain the names of each of the RSS feeds.
Line 5. Set’s the rssDisplay to contain the titles from teh first feed.

Final Summary

Scala has been an extraordinarily interesting language. The XML package alone makes it worth adding to your toolbox and, as can be seen by how clean the GUI design code is, it’s not a one trick pony. The type system, which sadly I haven’t been able to go into nearly enough detail here, has many of the nice features you’ll find in Haskell. And of course it’s great to be able to make use of all of Java’s libraries for free.

I do have a couple of small complaints:

  1. The error messages are sometimes a bit obscure, but this is a common flaw in all but the most mature open source projects.
  2. When code compiles, each anonymous function gets its own class. For anyone coding in a functional style, this quickly leads to a deeply cluttered directory.

These complaints are minor though, and the first can certainly be fixed. Overall it’s been a pleasure to work with and I look forward to using it for real tasks in the near future.

Coming Up

Next week I’ll be starting on Prolog. Prolog is a declarative language that is centered on formal logic.

Resources

Scala 3/4: XML

This is the third article in a series, to read the whole series go here.

Writing an RSS Aggregator with Scala

One of the cooler things about Scala is its XML processing and production features. Scala has several powerful mechanisms for creating DSL’s (Domain Specific Languages) which are essentially special purpose languages inside the parent language. The details of how it goes about this are beyond the scope of this article, but I’ll show you one way you can enjoy some of the results.

To illustrate how Scala can be used to process and generate XML, I’ll be putting together a simple RSS aggregator. It will print to the screen and write an html file with the channel titles and article titles.

Files Used in this Project

Libraries

import scala.xml._
import java.net.URL
  • scala.xml: Scala’s XML library has a vast number of features which I’ve only begun to sample.
  • java.net.URL: Java’s library for URL’s and HTTP connections.

I’ve mentioned that Java libraries can be called as though they were Scala libraries elsewhere, this is the first time I’ve actually done it. It’s painless as advertised.

Main Functions, Arrays and Command Line Arguments

object RssReader {
  def main(args : Array[String]) : Unit = { 
    val rssFeeds = combineFeeds(getRssFeeds(args(0)))
    printFeeds(rssFeeds)
    writeFeeds(rssFeeds)
  }

One piece of syntax that is especially confusing in Scala is how it indicates that one type contains another, in this case Array[String]. This syntax looks a lot like what you’ll see for array indexing in most other languages. In Scala arrays are treated as functions that receive an integer argument to return their values, eg args(0) returns the first argument of args. This is something of a simplification, but it helps me remember what the syntax is.

Anonymous Functions, Maps and Folds

  def getRssFeeds(fileName : String) : List[(String, Seq[String])] = {
    // Given a config file containing a list of rss feed URL's,
    // returns a list with ('channel name', ['article 1', 'article 2'])
    var baseList : List[(String, Seq[String])] = List()
    getFileLines(fileName).map(extractRss).foldLeft(baseList)
        { (l, r) => l ::: r.toList }
  }

Anonymous Functions

The function on line 4, (l, r) => l ::: r.toList is a case of an anonymous function in Scala, I’ll break it down: 1. (l, r) =>: The arguments section, if I wanted to be pedantic I could specify their type, but anonymous functions already clutter code and their purpose should usually be pretty obvious. 2. l ::: r.toList: l is an already existing list object which is being prepended to r after r is turned into a list.

Higher-order functions

Scala’s for syntax is extraordinarily powerful, but sometimes it’s more than you need. The three most common higher-order functions are:

  • map: Apply a function to each element in a sequence.
  • filter: Use a boolean function to filter the elements of a sequence.
  • fold(generally split into foldLeft and foldRight): Use a 2-arity function to combine all of the elements in a sequence into 1 element.

The for syntax combines the functionality of map and filter, so is most useful when you need to do both to a sequence. It does not cover the functionality of fold. Here I’m:
1. Getting a sequence which contains the URL’s of several RSS feeds. 2. Mapping a extractRss over them to get the RSS data. 3. Folding the anonymous function (l, r) => l ::: r.toList over them to change them from immutable Seq values to Lists and combine them.

Filter and Some Sugar

  def combineFeeds(feeds : List[(String, Seq[String])]) 
      : List[(String, Seq[String])] = {
     // Cleanup function, given a list of feeds it combines duplicate channels.
    def combinedFeed(feedName : String) : Seq[String] = {
      feeds.filter(_._1 == feedName).map(_._2).flatten
    }

    val feedNames = feeds.map(_._1).distinct

    feedNames.map(x => (x, combinedFeed(x)))
  }

The filter function in combineFeed makes use of the fact that it is in combineFeeds‘s namespace and doesn’t need to have feeds passed to it. It filters through feeds, keeping those with the same name. It then passes the result to map which extracts the article lists, and flatten combines each list. Generally you would either pass a named function with a single argument to filter and map, or an anonymous function with the format (x) => x == feedName. This pattern comes up so often that Scala allows us to ignore the usual anonymous function syntax and use the underscore as a placeholder for a single variable.

Brackets are Optional in Single Liners

  def getFileLines(fileName : String) : Array[String] = 
    // Extracts the URL's from a file and breaks them up into individual strings.
    scala.io.Source.fromFile(fileName).mkString.split("\n") 

Opening Network Connections and Reading XML

  def extractRss(urlStr : String) : Seq[(String, Seq[String])] = {
    // Given a URL, returns a Seq of RSS data in the form:
    // ("channel", ["article 1", "article 2"])
    val url = new URL(urlStr)
    val conn = url.openConnection
    val xml = XML.load(conn.getInputStream)

    for (channel < - xml \\ "channel") 
      yield {
        val channelTitle = (channel \ "title").text
        val titles = extractRssTitles(channel)
        (channelTitle, titles)
    }
  }

Opening Network Connections

For practical purposes Lines 2-4 are straight Java. They format a URL, open a connection, download the stream and convert it into the XML library's internal format.

Reading the XML

Line 6 is where I start making good on my promise of easy XML processing. xml \\ "channel" descends through the XML tree and extracts every element named "channel", we then use the for expression to take each of those elements and treat them each as XML trees in their own right. The \\ will descend through the entire tree, while \ only examines those nodes that are immediately under the given root.

  def extractRssTitles(channel : Node) : Seq[String] = {
    // Given a channel node from an RSS feed, returns all of the article names
    for (title < - (channel \\ "item") \\ "title") yield title.text
  }

In extractRssTitles you can see how \\ can be nested to do more complex dives into an XML document.

Display

To Screen

  def printFeeds(feeds : List[(String, Seq[String])]) : List[Seq[Unit]] = { 
    // Given a list of ("channel", ["article 1", "article 2"]), prints
    //  them each to screen.
    for (feed < - feeds) yield {
          println("*** " + feed._1 + " ***")
          for (title <- feed._2) yield println("\t" + title)
    }
  }

printFeeds takes a list of the feed objects and displays them on screen. There are a couple of ways to extract data from a tuple. If you're dealing with a complex set of structures, you can use match/case. I knew I'd always be dealing with a 2-tuple, so I chose to use the _# syntax, where _1 will give you the first value in the tuple, etc.

To HTML

  def writeFeeds(feeds : List[(String, Seq[String])]) = {
    // Given a list of ("channel", ["article 1", "article 2"]), generates
    //  and writes an HTML document listing the articles with channel names
    //  as headers.
    val html = 
      
        {for (feed < - feeds) yield
          

{feed._1}
    {for (title < - feed._2) yield
  • {title}}
} XML.saveFull("rss.html", html, "UTF-8", true, null) } }

When you produce XML in most languages you have to choose between using hard to read code that produces guaranteed well formed XML, or generating your XML out of strings that are easy to read but you can't be sure is well formed.

This is what has had me excessively excited about Scala for the past week. It seems that the majority of projects I get involved in sooner or later involve doing some kind of work with XML, and it's usually one of the ugliest parts of the job. Here you can see I have XML freely interspersed with the rest of my code, and was by far the easies part of what I wrote this week. At any time inside the XML, if I need to break back out into Scala I just surround the statement I want to use in curly braces. With it I've been able to clearly produce XML that I can be confident is well formed.

My Experience

Arrays

An unpleasant surprise I got while I was working on this program was that, unlike other sequence types, Arrays in Scala have a simple way to convert to Lists.

Functional Programming

Reading over my own code, I realize that functional style has become a habit. It'll be interesting to see how well I cope when I try to pick up a language where functional style isn't a realistic option. To anyone reading, if there are places in my Scala code where it would be more clear if I used a more object oriented style, please let me know.

XML

One more time. The XML processing capabilities of this language are beautiful. Scala is going to end up being part of my standard arsenal for this reason alone.

Resources: