In the past couple of months I’ve started keeping my notes in a personal wiki using Vim Wiki. Rather than just keeping them on my drive, I decided to share what I’ve figured out. Right now most of what I have is links and a bit of sample code. When I do a major update I’ll post a link here. I hope you find it useful.
New Plans
I have a couple of announcements to make. First, I haven’t been able to produce anything substantial this week because I’ve had a severe respiratory bug that’s kept me out of commission since Monday. If the Doctor’s correct I should be up and rolling again in the next couple of days. Second, I’m going to be making some changes to the content of this blog. When I started most of my other projects had been lying fallow for months. Now three of them have picked up at once, all of which require that I learn specifically for them. This is great, it’s exciting, it means that I have to choose between them, the blog, and sleep.
Rather than drop the blog completely I’m going to be writing about what I learn as I’m working on each of these projects, right now the things in my queue are:
- Relational DB Design
- The Wintermute Engine
- Prolog (More advanced than a month’s worth)
- Scala (see Prolog)
- Lift
- Pencil
I’ve enjoyed working on learning new languages and sharing the projects as I’ve gone, and would like to continue to do so. So I intend to, either at a much slower schedule or intermittently as I have time, continue to do the new language project. I’d be interested in feedback from any current readers for what they’d like to see.
Smalltalk 1/4: Getting Started
This is the first article in a series, to read the whole series go here.
The Plan
You’ve heard about it, every child will have a portable computer, small enough to carry in her bookbag. Its software will be an open environment that they can change and program themselves. I’m talking, of course, about the Dynabook, from Alan Kay’s proposal in 1972. The programming language that was to be the center of the Dynabook and its environment was Smalltalk, one of the first object oriented programming languages. Smalltalk was intended to be easy for children to learn, to that end has a simple and highly regular syntax and a deeply interactive environment.
Dynabook was an idea before its time. It lives on in spirit in the OLPC project, which also contains Smalltalk, or an application of it in the form of Squeak’s EToys.
Modernly Smalltalk has a number of implementations. One of the best known is Squeak which Alan Kay continues to be involved with. Squeak is an wonderful programmer’s playground where Kay and his team have tried out a multitude of ideas, still with the goal of creating an environment that children may learn in. Unfortunately for the working programmer the playground has become rather cluttered with toys, so the Pharo team has made a stripped down version for a much leaner programming environment.
As I alluded to above, Smalltalk isn’t just a programming language, it’s an environment in its own right. When you open up the Pharo image you have an entire integrated development system, including the System Browser where you do most of your coding, the debugger and the unit test system. And all of which is written in Smalltalk and can be modified inside the system in real time.
What to Expect
When this article is complete:
- You will have:
- An installation of Pharo
- An object that says Hello
- You will know:
- Basic elements of the Pharo/Smalltalk environment including:
- The Workspace
- The Transcript
- The Class Browser
- How to create new classes in the Pharo environment
- Basic elements of the Pharo/Smalltalk environment including:
What You’ll Need
To install Pharo, start by downloading the zip of latest version from Pharo Downloads. You’ll have the choice between the Cog and Standard VM. Cog is newer and faster, which won’t matter much for our purposes, but why not get the fancier toy when you can?
Unzip “Pharo.app” to the location you want to keep it, “Program Files” will do the job. Once you’ve unzipped it, navigate into the Pharo.app directory and double click the “Pharo” link and your new Pharo system will open.
A Brief Tour of the Environment
Smalltalk was designed to be an entire environment, separate from whatever system that it’s running on. The advantage is it gives you a completely portable programming environment that you can move to any machine that has a Smalltalk VM and it will work the same way on each of them. A disadvantage is that there’s a lot to get used to in it. For the next couple of weeks, 90% of the work is going to be in the System Browser with side trips in a few of the other tools. The major components of it are:
- System Browser: This is where you’ll do your actual coding.
- Packages: Packages are similar to modules, but they only have formal existance when you’re importing them in the system. Once they’re part of your Smalltalk image, their purpose is purely organizational.
- Classes: Once you’ve selected a package, the classes are the primary element in the system.
- Protocols: Protocols, like packages, are a way to organize the methods in a class.
- Methods: Methods are what the classes do.
- Edit Pane: The Edit Pane is where you’ll do most of your coding including specifying classes and adding methods.
- Workspace: You can view the Workspace as a scratch pad, you’ll use it when you’re trying to figure something out in the small, or when you’ve completed a piece of work and are doing a bit of ad hoc testing.
- Transcript: The transcript is where you’ll print logs and quick bits of system interaction. You can think of it as a combination of stderr and stdout in a more conventional system.
Files Used in this Project
- Virtuous-Demo.st: Contains all of the sample code developed during this project.
To import “Virtuous-Demo.st” into your Smalltalk image:
- Open Pharo
- Click on the background to open the World menu.
- In the World menu select Tools > File Browser
- In the File Browser dialog, navigate to the folder that contains “Virtuous-Demo.st”
- Select “Virtuous-Demo.st”
- Click the “install” button at the top of the File Browser dialog.
- Close the File Browser.
You can verify that the package has been correctly installed by doing the following:
- In the World menu select System Browser
- In the Package pane, scroll to the bottom where you should see a package called “Virtuous-Demo” which contains the class “Greeter”.
To keep it as part of the image (along with any other changes) click “Save” in the World menu or to exit click “Save and quit”.
Executing Code
To execute code in Pharo:
- In the World menu, select Workspace, this is where you’ll enter the code to execute.
- In the World menu, select “Tools” > “Transcript”, this is where results will be displayed.
In the Workspace window enter the following code:
Transcript show: 'Hello World'; cr.
There are two ways to run the code. You can either right-click on the same line as the code and select “do it” or fellow keyboard addicts can type Alt-d while the cursor is on the same line as the code. In either case the words “Hello World” will then appear in the Transcript followed by a new-line. You might also see some paired numbers that come up during the process, this is a side effect of the Transcript being where logging data for the system in general is sent.
Understanding the code you just Executed
One element at a time:
Transcript
is the object of interest, generally objects will start with a lowercase letter, but this is a special caseshow:
is a method (message) that is called (sent) to theTranscript
object. This says that whatever object it is given should be written to the Transcript window, the:
indicates that it takes an argument which is…'Hello World'
is a string, note that Smalltalk strings are surrounded by single, not double quotes.;
indicates the end of one method, but that the following method is also getting sent to the same object.cr
another method that is sent to Transcript and writes a new line to the Transcript window..
indicates the end of a statement.
The Code
Coding in Smalltalk
The first step you’ll take in your Smalltalk project is to add it to the Packages pane of the System Browser window. To do this:
- Make sure that the Packages pane is selected, left-clicking it will do the trick.
- Right-click in the Packages pane and select “add category…”
- Enter a name that works for you and click OK. (Mine was “Virtuous-Demo”, remember, this is just to keep your code organized)
Now that you’ve added your category, select it from the Packages pane, notice that in the Edit Pane there’s a template for creating a class:
Object subclass: #NameOfSubclass
instanceVariableNames: ''
classVariableNames: ''
poolDictionaries: ''
category: 'Virtuous-Demo'
You’ll want to add a new class:
- Change ‘#NameOfSubclass’ to ‘#Greeter’
- Right click in the Edit pane and select “accept” (or type alt-s) to write your class to the system.
Now you can see the new class in the Classes pane. In the Protocols pane click “as yet unclassified”, this is a generic bucket for any methods you write without sorting them. Now you can see a template for a method in the Edit pane:
messageSelectorAndArgumentNames
"comment stating purpose of message"
| temporary variable names |
statements
Start writing your method, since there’s going to be no need to track temporary data within an object I made this a “class” rather than “instance” method, so click the “class” button underneath the class pane. Now change the Edit pane like so:
sayHello
"Greet the User"
Transcript show: 'Hello World'; cr.
The first line is the method name, followed by a comment. I removed the | temporary variable names|
because there was no need for them, and put the actual action of the method in the last line.
When you’ve finished entering the method “accept” it the same way you did with the class. When you “accept” a new method for the first time, it will ask you to enter your full name. Smalltalk automatically tracks all changes to the system and it will uses each individual’s name to track who has made the changes.
To try your new method out, type Greeter sayHello.
into the Workspace and then execute it.
Arguments in Methods
To get an idea of how arguments, I wrote a version of sayHello
that knows who it’s talking to. To add another method click “as yet unclassified” in the Protocols pane again and the Edit pane will be populated with the template again, add the method sayHello:
as below:
sayHello: toWho
"Greet the User"
Transcript show: ('Hello ', toWho); cr.
Inside the method declaration is the argument list. To indicate that a method has an argument append a :
to it and follow it with the argument name. In the body of the method there’s one more bit of Smalltalk syntax the ,
operator concatenates two strings. Strictly speaking it’s not Smalltalk syntax, but just an operator attached to the String class. I’ll get into the details of operator declaration in a later article.
Try this method by typing Greeter sayHello: 'Professor'.
Final Summary
Nice things about Smalltalk so far:
- Very simple syntax
- Very clear code
Difficult things about Smalltalk:
- Learning the environment
- Resisting the urge to spend hours just playing with the system instead of working
Coming Up
Next week I’ll be putting together a class for calculating the “FizzBuzz” problem and unit test it using Pharo’s integrated unit test system.
Resources
- Squeak: Squeak Smalltalk is the basis of many of the modern free Smalltalk implementations
- Pharo: Pharo is a professional design system based on Squeak
- Smalltalk Zen: Dimitri Zagidulin’s informative page about web development in Pharo/Squeak
- Smalltalk Language Notes: A nice reference for a lot of basic Smalltalk programming
Robot Framework
Test Automation with Robot Framework
The Plan
Robot Framework is a functional test framework that makes it possible for its users to:
- Write tests in easily understood language
- Organize what tests are run for specific purposes
- Generate both high level and detailed test results
Notice that what it does not do is any specific testing activity. Instead it acts as a front end for libraries like Selenium and AutoIT which perform the actions themselves. For the purpose of this tutorial I’ll be focusing on web automation using Selenium as the back-end.
What to Expect
When this article is complete:
You will have:
- An installation of Robot Framework with Selenium
- Several tests for a basic login page
You will know:
- How to use Robot Framework(RF):
- To drive Selenium
- To write functional tests
- To write data-driven tests
- To select or exclude tests by their tags
- How to read an RF results file
- How to read an RF log file
- To find identifiers on an HTML page
- How to use Robot Framework(RF):
Installing Robot Framework and Selenium
Before installing Robot Framework, you need to install Python 2.5 or later. Robot Framework hasn’t been updated to work with Python 3, so currently 2.7 is your best bet. If you don’t already have Python installed you can download it from here.
Next you’ll need to download Robot Framework and Selenium. For each of the installs select the default values.
Finally, add your python installation and robot installation to your path. If you’ve installed Python 2.7 it will be: “c:\Python27\;c:\Python27\Scripts\;”.
Files Used in this Project
Executing Robot Framework Tests
In Robot Framework each file contains one or more tests. That file is treated as a test suite. Every directory that contains a test suite file or directory is also a test suite. When RF is executed on a directory it will recurse through all files and directories of the correct kind except those that start with an underscore character.
To run the tests in “SimpleTest.txt” navigate to the directory that contains it in the command line and type:
pybot SimpleTest.txt
There are arguments that will give you greater control over what tests are executed, but I’ll get to those later in this article.
Notice that while the test is running there are two windows. One is the window in which the test is being executed, the other is the controller window that Selenium is executing the JavaScript that controls the other window from.
When the test is finished executing four files will have been generated: report.html, log.html, output.xml and selenium_server_log.txt. Of these the only one’s you’ll generally need to be aware of are report.html and log.html.
Reading the Results
Your first stop after a test run is report.html. When you open it, if you’re lucky, you’ll see this happy image:
The background is green when all tests have passed, red if any have failed. If you need more detail than that the “Test Statistics” section gives how many tests have passed and failed sorted by criticality, tag and suite. The “Test Details” section gives how long the test took to run and, if it failed, what the fail message was, as can be seen here:
When a test fails, unless the message makes it immediately obvious, you’ll generally next need to read the log. If you click the failed test name, it will then open up the test logs:
The log gives a detailed view of the execution of each of the tests. While the information for all of the tests are available, the tests and steps that are failed are marked in red and already expanded.
The Code
Syntax and Formats
Robot Framework files are sorted into tables. The format of those tables is flexible, you may create them in tab-separated, html, ReStructured text, or just plain text. I’ve tried writing tests in each of these, HTML makes the prettiest files, and is also a pain to work with. In the end, plain text has been the best combination of nice to read for my customer and easy for me to maintain.
There are up to four tables in a test file, the “Testcases” table is mandatory, it contains all the tests you’re going to run, the optional tables are:
- Settings: Data needed across tests, like what libraries to use and how to setup and teardown tests.
- Variables: Variable names and values
- Keywords: These are function definitions, RF calls them keywords, there’s no difference.
The Test
Given a Login Page verify that when the correct username and password are supplied, it is possible to login.
A First Attempt
Settings
*** Settings ***
Library Selenium Library
Each Table is marked out by *** <table name> ***
and continues until the next table name is reached. The text version of the RF syntax is whitespace delimited. Each item under a table is flush to the left of the screen. If a keyword takes arguments then the follow it on the same line, separated by either a tab or two or more spaces. I generally space them out more than two for readability.
The Library
setting imports the named library into the test suite. The Selenium Library allows its users to control a web browser automatically and ideally suited to the task at hand.
Identifying Elements on a Web Page
When automating any system, you have to have a way for your test software to find the elements that you’re interacting with. Selenium does this through one of several methods, depending on how automation friendly the software you’re testing is. If it’s very friendly then every element on the page has either a unique name
or id
attribute. If it’s more hostile then you can still identify any element through its xpath. I recommend using xpath only as a last resort. It’s a powerful tool, but makes your code hard to understand.
I wrote the login page, so it’s very test friendly. For simpler pages like this, I general just look at the sourcecode of the page to find out how to identify the elements, if you look at the Login Page, you’ll find that the identifiers are:
Test Cases
*** Testcases ***
Login Should Succeed When the Correct Username and Password are Entered
Start Selenium Server
Open Browser http://zdiles.chaosnet.org/ ie
Maximize Browser Window
Input Text uname BUser
Input Text pwd TestPass
Click Button login
Page Should Contain Welcome
Close Browser
Stop Selenium Server
I’ve named this file “SimpleTest.txt”. Here’s a line By line look at it, I’ve informally labeled the below steps with their purpose: Setup, Action, Verification, Teardown
- A Testcases table contains one or more tests.
- The first line will be the name of the test as used in the reports and logs.
- Setup Task: To use Selenium a server has to be started that will receive commands and control a browser. Notice that keyword names can have single spaces in them.
- Setup Task: A browser is opened to the page given, by default Firefox is used. If specified as it is here it can also use Internet Explorer or Chrome.
- Setup Task: By default the browser window isn’t maximized. This fixes that.
- Test Action: The
Input Text
keyword enters text into a field, it takes two arguments, the identifier for the element and the text to enter. This fills the username field. - Test Action: The next
Input Text
call fills the password field. - Test Action: One of the nice things about robot framework is that its keyword names tend to be pretty obvious. The argument passed to it identifies the Login Button.
- Test Verification:
Page Should Contain
searches through all the text on the page and passes if it contains the given text and fails otherwise. - Teardown Task: Close Browser, this will shutdown the browser opened in line 4, if not run then it just leaves all the windows opened between tests, each test will ad more leading to clutter and confusion.
- Teardown Task: Shuts down the Selenium Server, if this isn’t done then it can occasionally lead to trouble when you try to start the Selenium Server in a future test.
Suite Settings
The test as it’s written will faithfully perform the tests given, but has a couple of problems. When a test fails Robot Framework stops running the test to save time and avoid performing activities on a system that’s in a bad state. This means that if the verification step fails, then our teardown tasks will never be executed. To deal this problem, and make neater more readable test cases there are settings at both the test and suite level to setup and teardown tests.
Setup/Teardown
*** Settings ***
Library Selenium Library
Suite Setup Start Selenium Server
Suite Teardown Stop Selenium Server
Test Setup Open Browser http://zdiles.chaosnet.org/ ie
Test Teardown Close All Browsers
Setup and Teardown can be set at both the test and the suite level. Each fires off at different points during a test run:
- Suite Setup: Fires once before any tests during the suite are run.
- Test Setup: Fires before each test is executed.
- Test Teardown: Fires at the end of each test execution, even when the test fails.
- Suite Teardown: Fires when all tests in the suite have completed, even when one of them has failed.
This slims our testcase down considerably:
*** Testcases ***
Login Should Succeed When the Correct Username and Password are Entered
Maximize Browser Window
Input Text uname BUser
Input Text pwd TestPass
Click Button login
Page Should Contain Welcome
But we still have the Maximize Browser Window
keyword, which really doesn’t improve the reader’s understanding of the intent of the test, and is likely to be used in the execution of any future testcase. Unfortunately the setup/teardown keywords can only take one keyword.
The Keywords Table
You can solve this by adding a keyword that calls both of these:
*** Keywords ***
Setup Test
Open Browser http://zdiles.chaosnet.org/ ie
Maximize Browser Window
And change the Test Setup
setting to use it:
*** Settings ***
Library Selenium Library
Suite Setup Start Selenium Server
Suite Teardown Stop Selenium Server
Test Setup Setup Test
Test Teardown Close Browser
Making Your Tests More Readable
*** Testcases ***
Login Should Succeed When the Correct Username and Password are Entered
Input Text uname BUser
Input Text pwd TestPass
Click Button login
Page Should Contain Welcome
Now that I’ve gotten rid of the steps that obscure the intent of my test, it’s time to make the purpose of the tests themselves more clear by creating keywords with more meaningful names than those given, so we’ll add a couple more keywords to our Keywords
table:
Enter Username [Arguments] ${username}
Input Text uname ${username}
Enter Password [Arguments] ${password}
Input Text pwd ${password}
Click the Login Button
Click Button login
Login Is Successful
Page Should Contain Welcome
Which introduces the use of settings in individual keywords. The syntax used for keyword and test settings(to be seen soon) is the same. A special element surrounded in square brackets, you can find a full listing of these in Robot Framework’s User Guide. Here I’m able to specify arguments that will be passed into the keywords from the test case that calls it. The syntax for variables is ${<variable name>}
and can contain spaces.
Now my testcase is considerably more readable:
Login Should Succeed When the Correct Username and Password are Entered
Enter Username AUser
Enter Password TestPass
Click the Login Button
Login Is Successful
The Power of Tags
While it’s useful to be able to run every test in your test suite at once, often you only need to run a small subset of them to perform a smoke test, or to test only a specific feature. To this end you can tag tests and suites to indicate their function.
The code below is the same as what I’ve constructed with one more test, which should fail every time, and a couple of tags (Keywords left out because they don’t change):
*** Settings ***
Library Selenium Library
Resource LoginKeywords.txt
Suite Setup Start Selenium Server
Suite Teardown Stop Selenium Server
Test Setup Login Test Setup
Test Teardown Login Test Teardown
Force Tags FunctionalTest
Default Tags ValidTest
*** Testcases ***
Login Should Succeed When No Username Is Given
[Tags] InvalidTest
Enter Username ${empty}
Enter Password TestPass
Click the Login Button
Login Is Successful
Login Should Succeed When the Correct Username and Password are Entered
Enter Username AUser
Enter Password TestPass
Click the Login Button
Login Is Successful
One Test level setting has been added, tagging the first test with InvalidTest
. At the suite level there are two more. Force Tags
ensures that every test in the suite will be tagged as a FunctionalTest
. Default Tags
set’s a test’s tags to ValidTest
unless the test has a tag setting of its own like the first test.
There are two benefits from adding descriptive tags to your tests. When you run your tests your test results will be sorted by them. This will frequently help determine where an individual problem is. Additionally you can determine which tests get executed. For example when you use:
pybot -i InvalidTest SimpleTest.txt
Will only include the tests with that tag, while:
pybot -e InvalidTest SimpleTest.txt
Will exclude any test with that tag.
Keyword Files
Keywords like Enter Username
will be used in nearly every test you write for this page. It’s often helpful to break up your test suites into multiple files, or even multiple directories. To avoid copy pasting the same set of keywords into every file, you can break them out into a separate resource file. The easiest way in our current case is to just cut everything from *** Keywords ***
on down and put it in a new file called “LoginKeywords.txt”
We’re getting to a point where this is well beyond a simple test, so while you’re at it rename “SimpleTest.txt” to “LoginTest.txt” and add one more line to the Settings section of LoginTest.txt, like so:
*** Settings ***
Library Selenium Library
Resource LoginKeywords.txt
Suite Setup Start Selenium Server
Suite Teardown Stop Selenium Server
Test Setup Login Test Setup
Test Teardown Login Test Teardown
Force Tags FunctionalTest
Default Tags ValidTest
Directories as Suites
I’m going to add more test files, so to organize them I’m going to create a directory “LoginTest” and put “LoginTest.txt” and “LoginKeywords.txt” in it. Now if I execute pybot from the directory above the “LoginTest” directory as:
pybot LoginTest
It will execute all of the test files in LoginTest.
Init Files
As stated above, both directories and files are treated as test suites by Robot Framework. You can create a Settings section for a directory by adding the file “__init__.txt” and putting a Settings table in it. Since I’m about to add more tests and they’re all going to need the same setup and teardowns, I’ll move a few more settings from “LoginTest.txt” to “__init__.txt” giving us:
*** Settings ***
Library Selenium Library
Resource LoginKeywords.txt
Suite Setup Start Selenium Server
Suite Teardown Stop Selenium Server
Test Setup Login Test Setup
Test Teardown Login Test Teardown
And now the Settings section of “LoginTest.txt” is:
*** Settings ***
Library Selenium Library
Resource LoginKeywords.txt
Force Tags FunctionalTest
Default Tags ValidTest
Data Driven Tests
Data driven tests are designed to repeatedly run the same test with changes in the data is being used for inputs. This allows the person writing the tests to create a large number of tests scenarios with minimal effort.
Robot Framework has a special setting called Test Template
that lets you create files that are dedicated to Data Driven test suites:
*** Settings ***
Library Selenium Library
Resource LoginKeywords.txt
Test Template Login Should Fail When
Force Tags DataDriven ValidTest
*** Testcases *** Username Password
The Password is Wrong AUser TestFail
The Password is Empty AUser ${empty}
Both Fields are Empty ${empty} ${empty}
*** Keywords ***
Login Should Fail When [Arguments] ${username} ${password}
Enter Username ${username}
Enter Password ${password}
Click the Login Button
Login Is Unsuccessful
Using Login Should Fail When
as the template means that each of the three testcases use it as their only keyword, and apply the arguments given to it. You can see on the first line of the Testcases table there are two columns set “Username” and “Password”. Robot Framework ignores anything in the first line of the table after the name, the column names are strictly for the reader’s benefit. The other new element is the ${empty}
variable, this indicates an empty string.
For this suite you can see I’ve added one more new keyword to “LoginKeywords.txt” called “Login Is Unsuccessful.” It is the negation of “Login Is Successful”:
Login is Unsuccessful
Run Keyword And Expect Error * Login Is Successful
Run Keyword and Expect Error
does exactly what it says. The second argument is the error expected, *
will allow for any error. The third argument is the keyword to run, then the follow arguments get passed to the keyword in the third argument. This is a useful way to define negative test cases.
Variables
When your tests get even moderately complicated there’s going to be data that needs to change from one run to the next or depending on the context you’re running it in. Use a variable table to minimize the pain of dealing with this. For my tests I have two variables that I’m concerned with, the URL under test and the browser used, pulling these out as variables in the keyword file gives me:
*** Variables ***
${login address}= http://zdiles.chaosnet.org/
${browser}= ie
*** Keywords ***
Enter Username [Arguments] ${username}
Input Text uname ${username}
Enter Password [Arguments] ${password}
Input Text pwd ${password}
Click the Login Button
Click Button login
Login Is Successful
Page Should Contain Welcome
Login is Unsuccessful
Run Keyword And Expect Error * Login Is Successful
Login Test Setup
Open Browser ${login address} ${browser}
Maximize Browser Window
Notice that when they’re assigned the variable names have an =
appended to them, this is optional, but it makes the code a little more clear.
Final Summary
I’ve found Robot Framework to be powerful tool for designing and executing tests. By adding functional tests to your project you can improve its overall quality. Enjoy.
Coming Up
Next week I’ll start with Smalltalk, using the Pharo environment which is redesigned from Squeak to be a powerful professional development tool.
Resources
Prolog 4/4: GUI
Prolog 4/4: GUI
The Plan
I will write a GUI for last week’s RSS reader in order to explore how to write a GUI in a logic based language using Prolog’s XPCE toolkit.
The XPCE toolkit is:
- An IDE for SWI-Prolog.
- A library for writing GUI in Prolog.
- An object layer for Prolog.
What to Expect
When this article is complete:
- You will have:
- A GUI front end for the RSS reader from last week
- You will know:
- How to use SWIPL-Win.
- How to write a GUI in Prolog.
Files Used in this Project
- rssGui.txt: The GUI code. (Change the extension from txt to pl)
- rss.txt: The original RSS reader file, used here as a library. (Change the extension from txt to pl)
- feeds.txt: Sample RSS feed config file.
The Code
Compiling the Code… More or Less
I discovered that compiling GUI code for XPCE is easier said than done. I made several attempts, read a couple of tutorials and got nowhere. If there are any Prolog guru’s reading who wouldd like to set me straight, please leave a comment.
On the other hand, running the code on the is pretty easy:
- Download the files for this project and re-name them as described above.
- Double click on “rssGui.pl”
- In the SWIPL window, type,
rssGui.
At that point, you should have a window with the RSS reader in it.
Loading Libraries And Other Activities
:- consult(rss).
:- pce_autoload(finder, library(find_file)).
:- pce_global(@finder, new(finder)).
To load the rss
library from last week, tell Prolog to consult
it. The two lines afterward load the file finder into memory. This makes it possible to open a file finder to select the rss config file.
Laying Out the GUI, and Adding Hooks
rssGui :-
new(Frame, frame('RSS Reader')),
new(RssFeeds, browser),
send(Frame, append(RssFeeds)),
send(new(NewsList, browser(news)), right, RssFeeds),
send(RssFeeds, select_message, message(@prolog, newsDisplay, RssFeeds, NewsList)),
send(new(Buttons, dialog), below(RssFeeds)),
send(Buttons, append(button(load, message(@prolog, load, RssFeeds)))),
send(Frame, open).
XPCE makes it possible to access objects in Prolog by adding three predicates:
new
: Instantiates a new object.send
: Sends information to the object.get
: Gets information from an object.
To make a new GUI window, first a frame
is created(Line 2) which contains all the GUI elements.
Lines 3 and 4 create a List Box to contain the channel names and adds it to the Frame.
Line 5 adds a List Box to the right side of the channel list box for displaying news titles.
Line 6 adds an action so that when a value is selected in RssFeeds, the newsDisplay
predicate is fired with RssFeeds and NewsList for arguments.
Line 7 adds a container for a button.
Line 8 adds a button to the dialog which has the text “load” and fires load
when clicked.
Line 9 opens the frame.
Read the Config and Load the RSS Feeds
load(Browser) :-
get(@finder, file, exists := @on, FileName),
readConfig(FileName, URLs),
send(Browser, clear),
foreach(member(URL, URLs), readAndListChannels(Browser, URL)).
Line 2 opens up a standard file dialog and unifies the selected file with FileName
, from there the code is much like the display code from last week.
I used a separate predicate to write each of the URLs to avoid potential logical contradictions in the foreach
.
Fetch the Feed and Add it to the RssFeeds Browser
readAndListChannels(Browser, URL) :-
catch((rssFetch(URL, XML),
rssRead(XML, RSS),
foreach(member(channel(Name, News), RSS),
send(Browser, append, create(dict_item, Name, Name, News)))),
_,
readAndListChannels(Browser, URL)).
Again, much of this is similar to the code to display the feeds on the command line. The only real change is in Line 5, where the data from each channel is wrapped into a dictionary item with:
- identifier: Name
- key value: Name
- object: News
As before, this will keep trying until it succeeds or blows the stack.
Display the News Titles in the NewsList Browser
newsDisplay(RssFeeds, NewsList) :-
get(RssFeeds, selection, Channel),
get(Channel, object, News),
get(News, size, SizeVal),
send(NewsList, clear),
foreach(between(1, SizeVal, LineNumber),
sendVector(NewsList, News, LineNumber)).
Line 2 unifies Channel with the currently selected item in RssFeeds, the dictionary item from readAndListChannels
.
Line 3 pulls the object from the dictionary item, a vector.
Line 4 determines the length of the vector.
Line 5 clears the NewsList so we don’t just keep adding to the end.
Line 6-7 iterates over each of the values in the vector and appends them to NewsList with sendVector.
Append a Cell from a Vector to a Browser
sendVector(Browser, Vector, Index) :-
get(Vector, element, Index, Value),
send(Browser, append, Value).
Given a Browser, Vector and Index, sendVector
pulls the Indexth element from the Vector and appends it to the Browser.
Final Summary
Prolog’s logic based semantics make for an interesting programming experience. Up to this point I had trouble imagining how you’d deal with dynamic situations, like user input or a file being read in, in context to a language that is based on logic. Now that I’ve spent a month working with it, it’s an incredibly elegant way to solve problems. That being said, I think I’m going to stick with Python for my day to day scripting needs.
Coming Up
Next week is a fifth Monday, so I’ll be working with a tool, Robot Framework. Robot Framework is a Python based glue language for writing test cases in a clear format which is readable to non-technical users.
In February, I’ll be working with Squeak. Squeak several neat features, it is:
- generally considered the first full object oriented language
- where unit tests frameworks came from (in the form that most of us are used to today)
- contained entirely in its own design environment
- designed to be useful as an education language
- host to Seaside a powerful web framework
Resources
- XPCE’s Main Page
- XPCE UserGuide: An excellent resource, it has a vast number of examples, but is also quite dense.
- Making an executable for XPCE
- FAQ for XPCE, executables
Prolog 3/4: XML Parsing
Reading, Parsing and Displaying RSS Data in Prolog
This is the third article in a series, to read it from the beginning go here.
The Plan
This week is when I really put the assertion that Prolog is a general purpose language to the test. Most decently rational Prolog programmers will tell you to do things like XML parsing and display through another language, then use Prolog dll’s to do the interesting logical stuff. But, it does have the libraries, so let’s see how they work.
What to Expect
When this article is complete:
- You will have:
- A command-line RSS aggregator.
- You will know:
- How to read a text file.
- How to read from an HTTP stream.
- How to parse an XML file.
- How to pretty print text to the screen.
Files Used in this Project
Compiling
To compile “xml.pl”, navigate to the directory that contains it on the commandline, then execute:
swipl -g main -o xml.exe -c xml.pl
The Code
Libraries Used
:- consult(library(sgml)).
:- consult(library(http/http_open)).
- sgml: The SGML library, which also handles XML.
- http/http_open: The HTTP library handles server and client code, I just used the small bit that let’s me treat an HTTP stream like a file.
Reading a File
readConfig(Lines) :-
open('feeds.txt', read, ConfigStream),
read_stream(ConfigStream, Lines).
read_stream(ReadStream, Lines) :-
read_line_to_codes(ReadStream, Line),
read_stream(ReadStream, Line, Lines).
read_stream(ReadStream, end_of_file, []) :- close(ReadStream), !.
read_stream(ReadStream, OldLine, [Atom | Lines]) :-
read_line_to_codes(ReadStream, NewLine),
string_to_atom(OldLine, Atom),
read_stream(ReadStream, NewLine, Lines).
read_stream
is a helper function to read multiple lines from a stream. It reads a stream in one line a time and returns/unifies its contents to Lines
. Most of it is similar to what you saw in the previous article, recursively pulling off one line at a time, testing if it’s the last line and adding it to the list. The two key elements that are worth noting are: the use of the atom end_of_file
, which is what the file stream returns when the stream is empty and the use of !
to indicate that if execution makes it to closing the stream, then there’s no need to backtrack.
readConfig
‘s functionality is pretty obvious. Open a file using an atom with the file’s name. Remember, single-quotes are for atoms, double quotes are for strings. This opens a read only stream and unifies it with ConfigStream
, then read_stream
pulls the text in the file out into Lines
.
Reading From HTTP
rssFetch(URL, XML) :-
http_open(URL, XmlStream, []),
load_xml_file(XmlStream, XML),
close(XmlStream).
The pattern here is similar to the one for readConfig, except that it extracts the XmlStream
straight to a Prolog XML structure. There’s quite a bit to it, but the short version is, it’s a nested linked list. There are a variety of functors in that list, but the one that interests us is element
which has the form element(<element name>, <attributes>, <sub elements>)
. For example when given the xml:
in text
load_xml_file
returns:
[element(testOuter, [],
['\n ', element(inside, [inAtt=hello], ['in text']), '\n'])].
Parsing XML
rssRead([], []).
rssRead([element(channel, _, Elements) | _], [Rss | Rsses]) :-
channelRead(Elements, Rss), rssRead(Elements, Rsses).
rssRead([element(rss, _, Elements) | _], Rss) :-
rssRead(Elements, Rss).
rssRead([_ | Elements], Rss) :-
rssRead(Elements, Rss).
channelRead(Elements, channel(Name, Titles)) :-
titleRead(Elements, Name), itemsRead(Elements, Titles).
itemsRead([], []).
itemsRead([element(item, _, Elements) | Items], [ Title | Titles ]) :-
titleRead(Elements, Title), itemsRead(Items, Titles).
itemsRead([_ | Items], Titles) :-
itemsRead(Items, Titles).
titleRead([element(title, _, [Text | _]) | _], Text) :- !.
titleRead([_ | Elements], Text) :-
titleRead(Elements, Text).
Strictly speaking Prolog is a weakly typed language, you have only a couple of basic types which all reduce to atoms and functors. But with those you can do the same tricks that are generally associated to algebraic type systems. Here is one example where I use the element
functor to recurse through the XML structure and extract the titles.
Once you know how Prolog’s XML structures work, it’s just a matter of figuring out what information you need and recursing until you get it. For this purpose I’ve created my own functor channel(<channel name>, <title list>)
.
In Prolog functors and atoms are created simply by using them. This makes writing the code feel a lot like freeform sketching a picture. I like it. But I’m pretty sure that unless I made an effort to document my code well, it would make large projects unwieldy.
Pretty Printing Text
displayRss(Channels) :-
foreach(member(channel(Name, Titles), Channels),
(writef("*** %w ***\n", [Name]), displayTitles(Titles), nl)).
displayTitles(Titles) :-
foreach(member(Title, Titles), writef("\t%w\n", [Title])).
writef
works similarly to printf in nearly every language I’ve worked in. It has a few quirks that are detailed in its docs.
Main And Helper Text
readAndDisplayRss(URL) :-
catch((rssFetch(URL, XML),
rssRead(XML, RSS),
displayRss(RSS)), _, readAndDisplayRss(URL)).
main :-
readConfig(FeedURLs),
foreach(member(URL, FeedURLs), readAndDisplayRss(URL)),
halt.
The above is glue code. It’s been written to continue retrying until it succeeds or runs out of stack, so make sure your URL’s are valid.
Final Summary
When I was reading up on Prolog, I heard predicates described as a generalization on functions. This week has driven that one home. Working in Prolog has felt a great deal like when I was first learning Haskell. I’ll spend hours fighting with something that I could do in my sleep in another language, then when I figure it out it is extraordinarily clear, obvious and feels cleaner than anything else I’ve worked with.
Hard Lessons Learned
foreach
is not the same as a for comprehension/list comprehension, no matter how much it looks like one. The key difference is that the second term in the statement does not contain its own namespace. What happens in there happens throughout the same namespace that contains the foreach
. For example:
main :-
foreach(member(Number, [1, 2, 3, 4]),
(Square is Number * Number,
write(Square), nl)).
Does not print the square of the numbers 1 through 4. It first asserts that Square is equal to 1 * 1, then that it is also equal to 2 * 2, 3 * 3, and 4 * 4, which is clearly false and ends execution of the foreach
. It is possible to do this, the safe and idiomatic way is to break the second term out into its own predicate:
showSquare(Number) :-
Square is Number * Number,
write(Square), nl.
main :-
foreach(member(Number, [1, 2, 3, 4]), showSquare(Number)).
That lesson took me half a day for me to learn. You’re welcome.
Coming Up
Next week I’ll be working out how to use XPCE, Prolog’s native GUI library.
Resources
Prolog 2/4: Loops, Decisions and Tests
Writing and Testing Fizzbuzz in Prolog
This is the second article in a series, to read it from the beginning go here.
The Plan
This week I’ve worked my way through solving the Fizzbuzz problem in Prolog. The most difficult part of working on it was learning to deal with Prolog’s syntax. Once I got a decent grasp of the syntax, the solution to the fizzbuzz problem fell out quite naturally.
What to Expect
When this article is complete:
- You will have:
- An executable that produces fizzbuzz up to 20
- Unit tests that verify several of predicates
- You will know:
- How to write recursive clauses.
- How to write conditionals.
- How to write loops.
- How to trow and catch exceptions.
- How to write and run unit tests.
Files Used in this Project
- fizzbuzz.txt (Rename to fizzbuzz.pl)
The Code
Basic Predicates
isFizz(Num) :- 0 is Num mod 3.
isBuzz(Num) :- 0 is Num mod 5.
isFizzbuzz(Num) :- isFizz(Num), isBuzz(Num).
There are two basic ways that information comes out of a predicate. The first is by filling in all of its arguments, then it returns either true or false depending on how they evaluate in the predicate’s body.
Conditionals
fizzbuzz(Num, Res) :-
isFizzbuzz(Num) -> Res = 'fizzbuzz';
isFizz(Num) -> Res = 'fizz';
isBuzz(Num) -> Res = 'buzz';
Res = Num.
Prolog’s conditional is ->
. a -> b
is equivalent to if a then b
. You can see above that the conditionals are separated out by Prolog’s “or” operator is ;
. In this context, ;
operates as an else. These aren’t special cases, but the standard operation of Prolog. For example:
- If isFizzbuzz(Num) evaluates to true, then Res = ‘fizzbuzz’ and the statement evaluates to true and the following ‘or’ statement is short-circuited.
- If isFizzbuzz(Num) evaluates to false, then the whole if statement evaluates to false and then the other side of the ‘or’ operator is evaluated.
Recursion and Exceptions
fizzbuzzes(TopNum, TopNum, List) :-
List = [],!.
fizzbuzzes(TopNum, CurrentNum, [Head | Tail]) :-
CurrentNum > TopNum -> throw('the CurrentNum is greater than TopNum');
TopNum < 1 -> throw('the TopNum is less than 1');
(NextNum is CurrentNum + 1,
fizzbuzz(CurrentNum, Head),
fizzbuzzes(TopNum, NextNum, Tail)).
Here I took advantage of the fact that Prolog evaluates its terms on the left side as well as on the right of :-
. For the first clause, if the first two terms have the same value then this clause is the one to fire. List is set to [], then all further evaluation stops because of the !
. I’m still a little vague on exactly how it works, but the short version is that if a predicate has multiple possible values, then Prolog will normally try to look for them. The !
tells Prolog that once it reaches it, it shouldn’t look any further.
Line 3 is in some ways the oddest one in the set. The first two values passed in aren’t unusual. The third value [Head | Tail]
is a list which doesn’t have a name as a whole, but its head and tail do. For our use this is will be the return value of the predicate, though in Prolog this isn’t entirely a meaningful phrase. Later you’ll see a case where the “return value” is passed into the predicate to verify if it’s valid.
In Line 4 I verify if the CurrentNum is greater than the TopNum, this should never happen since there’s a wrapper that’s intended to be used that won’t let it, but there’s no harm in a bit of verification. If it is then an error is thrown with the term: 'the CurrentNum is greater than TopNum'
. Normally terms don’t have spaces in them, but if you surround a term in single quotes it can have any combination of characters.
Clauses with More Than One Arity
fizzbuzzes(TopNum, List) :-
OneHigher is TopNum + 1,
fizzbuzzes(OneHigher, 1, List).
This fizzbuzzes
is the same clause as the one in the previous section, but with Arity 2 instead of 3. In Prolog these are differentiated as fizzbuzzes\2
and fizzbuzzes\3
. In the second line you can see some arithmetic, which is OneHigher = TopNum + 1
.
Loops
printFizzbuzzes(TopNum) :-
fizzbuzzes(TopNum, FizzbuzzList),
forall(member(X, FizzbuzzList), (print(X), nl)).
main :-
printFizzbuzzes(20),
halt.
I claimed in the last article that Prolog had no looping constructs. This is a half truth. It has no special syntax for loops, but it does allow for the creation of a for loop as a predicate. forall
is part of swipl’s standard library where the first argument is a generator for terms and the second argument is applied to each of those terms.
Unit Tests
Starting Unit Tests
:- begin_tests(fizzbuzz).
A clause that has a body, and not a head is a message to the compiler. A sequence of unit tests always starts with the being_tests
clause. The term inside it will mark out what test suite this is.
Test Cases, Negation
test(isFizz_isnot) :-
not(isFizz(5)).
When a predicate named test
is defined it will be added to the test suite. not
here is a little bit strange, it does not mean that something is false, but that it can’t be asserted as true. So if there is no assertion of the truth or falsehood of a statement, it will return true.
Not Really a Return Type
test(fizzbuzz_fizz) :-
fizzbuzz(3, 'fizz').
test(fizzbuzzes_empty) :-
fizzbuzzes(0, []).
test(fizzbuzzes_5) :-
fizzbuzzes(5, [1, 2, fizz, 4, buzz]).
Where previously I used the second argument in fizzbuzz
and fizzbuzzes
to return a value, here I’m using it to verify the truth of a statement.
Closing the Test Suite
:- end_tests(fizzbuzz).
end_tests
closes out the test suite.
Running Your Tests
To execute the unit tests, run swipl
from the directory that contains fizzbuzz.pl. Load fizzbuzz into the interpreter with [fizzbuzz].
, then run the tests with run_tests.
Final Summary
Fizzbuzz comes out well in Prolog. I’ve spent a lot more time time thinking I was fighting with the syntax when the reality was I don’t get the semantics. It’s the first time in a few years I’ve had to remind myself, “The compiler’s not broken, you just don’t know what you’re doing.”
Coming Up
Next week we’ll put the rubber to the road. Prolog’s pretty cool for algoritmic stuff, let’s see how it handles networking and XML.
Resources
Prolog 1/4: Getting Started
Getting Started with Prolog
The Plan
Prolog is rather unusual when compared to other programming languages. Rather than describing a sequence of steps as in a procedural language, it’s a declarative language that uses formal logic as its primary metaphor. Prolog is used predominantly in academic settings for language research and artificial intelligence. That being said it is a general purpose programming language and used in commercial settings where the system must be able to reason about complex data like logistics, language processing and data mining. Some examples can be found at SICStus’s Website.
Although it’s a general purpose programming language, generally other languages are used to do GUI and interface coding. There are GUI packages available though, so I’ll give one of them a try in Week 4. With a little luck we’ll see Prolog’s strength in Weeks 2 and 3.
What to Expect
When this article is complete:
- You will have:
- An installation of SWI-Prolog
- A program that’ll say “Hello World”
- You will know:
- How to compile a program in SWI-Prolog.
- How to use the interactive interpreter to make logical queries from a logic base.
- That Socrates is mortal.
Files Used in this Project
- helloworld.txt (Rename to helloworld.pl)
- socrates.txt (Rename to socrates.pl)
SWI-Prolog
There are a large variety of implementations of Prolog. From these I chose SWI-Prolog because:
- It works on Windows. (All projects on this blog do)
- It has a GUI toolkit. (necessary for week 4)
- It’s free, under the LGPL. (Please remember to include a link to SWI’s page if you use it for your applications)
- It has an interactive interpreter. (I like interactive interpreters)
Installation
- Download the latest version of SWI-Prolog from their download page.
- Run the installer accepting all the defaults.
- Add
"C:\Program Files\pl\bin"
to your PATH in environment variables. How to change your path.
The Code
helloworld.txt (Rename to helloworld.pl):
% Say Hello Prolog
main :-
write('Hello World!'), nl,
halt.
Comments
In Prolog comments are preceded by a %
.
Clause Structure
Prolog programs are composed of clauses separated by periods. A clause is divided into the head and the body with the format <head> :- <body>
or just <head>
. When just the head is given it’s treated like <head> :- true
. The body of a clause is a statement composed of either a term, or sequence of terms composed by operators and ending in a period.
In Prolog a ,
is equivalent to boolean and
. In the above, should any of the statements return false, then the others will not be executed. Each of the terms:
* write('Hello World!')
: Writes “Hello World!” to the screen.
* nl
: Writes a new line to the screen.
* halt
: Causes Prolog to shutdown.
Running/Compiling Your Code
SWI-Prolog has an interactive interpreter, an interpreter and a compiler.
The Interactive Interpreter
A session with the interactive interpreter looks like:
C:\>swipl
Welcome to SWI-Prolog (Multi-threaded, 32 bits, Version 5.10.2)
Copyright (c) 1990-2010 University of Amsterdam, VU Amsterdam
SWI-Prolog comes with ABSOLUTELY NO WARRANTY. This is free software,
and you are welcome to redistribute it under certain conditions.
Please visit http://www.swi-prolog.org for details.
For help, use ?- help(Topic). or ?- apropos(Word).
1 ?- write('Hello World!'), nl.
Hello World!
true.
2 ?- halt.
C:\Documents and Settings\fberthold\My Documents\My Dropbox\VirtuousProgrammer\Prolog\1 - Getting Started>
You can also interact with the interpreter through SWI-Prolog’s IDE. I’m a command line junky so haven’t played with it too much yet, but you can find it in the Start menu.
The Interpreter
To run your programmer through the interpreter type:
swipl -g main helloworld.pl
Where “helloworld.pl” is your Prolog program, commonly called a logicbase. “main” is what you’ve declared to be the main clause in your program.
The Compiler
You can compile it with:
swipl -g main -o helloworld.exe -c helloworld.pl
This will generate a windows executable “helloworld.exe” by compiling “helloworld.pl”.
Prolog’s Hello World
It’s a little hard to do Prolog justice from this first example, because the primary function of Prolog is to describe and evaluate relationships, which don’t come up in ‘hello world’. To give a little more flavor of what Prolog does, here’s a simple relationship:
socrates.txt (Rename to socrates.pl):
% All men are mortal, Socrates is a man.
mortal(X) :- man(X).
man(socrates).
Here we have two assertions. That men are mortal and that Socrates is a man. We can how use Prolog to draw a couple of conclusions. The first step is to load “socrates.pl” into the interactive interpreter with:
swipl socrates.pl
Here’s a sample session in which we can find that Socrates is mortal and that if you are mortal, you are Socrates (Add more facts to the logicbase if you want more mortals):
1 ?- mortal(socrates).
true.
2 ?- mortal(X).
X = socrates.
Prolog’s single data type is a term
. Prolog terms can be(with examples from the above code):
- atoms: socrates
- numbers: no example in the above code
- variables: X
- compound terms: man(socrates)
- where
man
is called a functor
- where
Summary
Prolog is unlike any programming language I’ve worked with to date. Not only is it’s model entirely different, but it’s syntax was designed before it was obvious Algol style syntax was going to predominate, so it’s syntax is more influenced by it’s logical roots than what is currently considered normal looking syntax.
Coming Up
Next week I’ll be putting Prolog through it’s paces. The fizzbuzz problem should be interesting to solve. It feels like it should naturally be able to deal with conditional statements, but it doesn’t have any direct looping facilities other than recursion. Unit tests on the other hand also feel like a natural fit.
Resources
- Prolog on Wikipedia
- Prolog Syntax and Semantics on Wikipedia
- SWI-Prolog
- Adventures in Prolog: A tutorial that uses writing an text adventure game to teach Prolog.
- Introduction to Prolog: A simple, but clear tutorial on Prolog.
Trivia
- Despite being based on an entirely different semantic set than other programming languages, Prolog is Turing complete.
- Prolog was created at the University of Aix-Marseille by Alain Colmerauer and Phillipe Roussel, collaborating with Robert Kowalski of the University of Edinburgh in 1972.
- If you enter
X.
at the interactive interpreter, SWI-Prolog gives Douglas Adams fans the answer.
Scala 4/4: GUI
Writing GUI Code in Scala
This is the last article in a series, to read the whole series go here.
The Plan
For the last week of Scala I’ve put together a GUI for week 3’s RSS aggregator using the Swing library. Scala’s Swing library is a fairly thin wrapper around Java’s Swing library. The primary difference between them is that Scala’s library takes advantage of its more powerful type system.
What to Expect
When this article is complete:
- You will have:
- A front end for the RSS agregator I wrote in week 3.
- You will know:
- How to write a basic Swing application in Scala.
Screenshot:
Files Used in this Project
- RssReader.scala: A library for reading RSS information, that I wrote in the previous project.
- RssGui.scala: The sourcecode for this project.
- feeds.txt: A sample input file.
The Code
Libraries
import RssReader._
import swing._
You can use the libraries that you’ve written in the same way you call any other Java class library. Remember that you must compile RssReader.scala before you try to compile RssGui.scala.
A Top Level Object
object RssGui extends SimpleSwingApplication {
The SimpleSwingApplication incorporates all of the basics that you need for a standard windowed application in order to: create, draw and destroy the window when it’s closed.
Container for RSS Data
var rssFeeds = List(("None Loaded", Seq("")))
This will hold the data for the RSS feeds. I’ve chosen to initialize it with "None Loaded"
and an empty string because it gives the GUI something to display until a config file is selected and loaded, and it let’s Scala know what the rssFeeds
‘s type is.
File Chooser
var configFileChooser = new FileChooser(new java.io.File("./"))
var configFileName = new String
The configFileChooser
is a FileChooser
that will open, defaulting to the current working directory. configFileName
will hold the value selected, the details for how this will work are in the “Defining Behaviors” section.
Layout
For the most part the code here stands for itself, which is wonderful. Once you understand the basics for creation and layout for one component, they’re all the same. As a result I’ve be described each new idea in detail when I encountered it and let the rest alone.
Channel Panel Construction
Channel Select
val channelLabel = new Label {
text = "Select a channel:"
}
val channelSelect = new ComboBox(rssFeeds(0)._1) { }
val channelSelectPanel = new BoxPanel(Orientation.Vertical) {
contents += channelLabel
contents += channelSelect
border = Swing.EmptyBorder(0, 0, 0, 30)
}
Lines 1-3: Create a new label with “Select a channel:” for its value.
Line 5: Create a ComboBox that is initialized to the first tuple value in the first list value: “None Loaded”
Line 7: Create a new panel that is filled top to bottom.
Line 8-9: Add the Label and ComboBox to the panel.
Line 10: Surround the panel in an empty border with 30 pixels on the right side (to separate it from the RssDisplay).
Rss Display
val rssDisplay = new ListView(rssFeeds(0)._2)
This creates a nearly empty ListView. It has one line with an empty string. Until it’s populated with something more complex, it won’t display at all.
Combining channelPanel
val channelPanel = new BoxPanel(Orientation.Horizontal) {
contents += channelSelectPanel
contents += rssDisplay
border = Swing.EmptyBorder(0, 0, 30, 0)
}
This creates a new panel with Orientation.Horizontal
so each element will be added left to right. The channelSelectPanel
and rssDisplay
are added to complete the channelPanel
.
Button Panel Construction
val setConfigFileButton = new Button {
text = "Set Config File"
}
val loadRssButton = new Button {
text = "Load Feeds"
}
val buttonPanel = new BoxPanel(Orientation.Horizontal) {
contents += setConfigFileButton
contents += loadRssButton
}
Create the two buttons and lays them out in a panel.
Top Window
val topWindow = new BoxPanel(Orientation.Vertical) {
contents += channelPanel
contents += buttonPanel
border = Swing.EmptyBorder(10, 10, 10, 10)
}
Add the two top level panels together and put a border around them to make it neat.
Defining Behaviors
The Last Bit of Layout
def top = new MainFrame {
title = "Rss Reader"
// Overall construction
contents = topWindow
This creates the overall window and assigns topWindow
as its contents. The title
“Rss Reader” will appear in the title bar of the window.
Listening
listenTo(channelSelect.selection, setConfigFileButton, loadRssButton)
Each of the GUI elements that you want to be able to have act as controllers must be listed in listenTo. I’ve chosen to add them all at once, you can also add them one at a time as in:
listenTo(channelSelect.selection)
listenTo(setConfigFileButton)
listenTo(loadRssButton)
To determine when a value has been selected we are listening to channelSelect.selection
instead of channelSelect
.
Reacting
For each action performed on one of the listenTo
‘ed elements it will run through each of the listed reactions to see if any of them are appropriate for the action/object combination.
reactions += {
case swing.event.SelectionChanged(`channelSelect`) =>
rssDisplay.listData = rssFeeds(channelSelect.selection.index)._2
pack
case swing.event.ButtonClicked(`setConfigFileButton`) =>
if(configFileChooser.showOpenDialog(channelSelectPanel) ==
FileChooser.Result.Approve) {
configFileName = configFileChooser.selectedFile.getAbsolutePath
}
case swing.event.ButtonClicked(`loadRssButton`) =>
if(configFileName != "") {
rssFeeds = combineFeeds(getRssFeeds(configFileName))
channelSelect.peer.setModel(ComboBox.newConstantModel(rssFeeds.map(_._1)))
rssDisplay.listData = rssFeeds(0)._2
pack
}
}
}
}
For readability I’ve left the entire set intact above, I’ll look at them for each case here.
A New Channel is Selected
case swing.event.SelectionChanged(\`channelSelect\`) =>
rssDisplay.listData = rssFeeds(channelSelect.selection.index)._2
pack
Line 1: The case looks for the event, SelectionChanged
and to which element that event is applied, channelSelect
, the backquotes above are important.
Line 2: When the selection changes, set the display to contain the story titles from the rssFeeds value with the same index as the value selected.
Line 3: Once the values have been changed, packs the window so everything fits.
The Config File Button is Clicked
case swing.event.ButtonClicked(\`setConfigFileButton\`) =>
if(configFileChooser.showOpenDialog(channelSelectPanel) ==
FileChooser.Result.Approve) {
configFileName = configFileChooser.selectedFile.getAbsolutePath
}
Line 2-3: Opens the configFileChooser and checks to make sure that a value has come back.
Line 4: Assigns the absolute path to the selected file to configFileName
.
The Load Rss Button is Clicked
case swing.event.ButtonClicked(\`loadRssButton\`) =>
if(configFileName != "") {
rssFeeds = combineFeeds(getRssFeeds(configFileName))
channelSelect.peer.setModel(ComboBox.newConstantModel(rssFeeds.map(_._1)))
rssDisplay.listData = rssFeeds(0)._2
pack
}
Line 2: Verifies that some value has been assigned to configFileName.
Line 3: Uses combineFeeds from RssReader to read in the RSS values.
Line 4: Set’s the channelSelect ComboBox to contain the names of each of the RSS feeds.
Line 5. Set’s the rssDisplay to contain the titles from teh first feed.
Final Summary
Scala has been an extraordinarily interesting language. The XML package alone makes it worth adding to your toolbox and, as can be seen by how clean the GUI design code is, it’s not a one trick pony. The type system, which sadly I haven’t been able to go into nearly enough detail here, has many of the nice features you’ll find in Haskell. And of course it’s great to be able to make use of all of Java’s libraries for free.
I do have a couple of small complaints:
- The error messages are sometimes a bit obscure, but this is a common flaw in all but the most mature open source projects.
- When code compiles, each anonymous function gets its own class. For anyone coding in a functional style, this quickly leads to a deeply cluttered directory.
These complaints are minor though, and the first can certainly be fixed. Overall it’s been a pleasure to work with and I look forward to using it for real tasks in the near future.
Coming Up
Next week I’ll be starting on Prolog. Prolog is a declarative language that is centered on formal logic.
Resources
Scala 3/4: XML
This is the third article in a series, to read the whole series go here.
Writing an RSS Aggregator with Scala
One of the cooler things about Scala is its XML processing and production features. Scala has several powerful mechanisms for creating DSL’s (Domain Specific Languages) which are essentially special purpose languages inside the parent language. The details of how it goes about this are beyond the scope of this article, but I’ll show you one way you can enjoy some of the results.
To illustrate how Scala can be used to process and generate XML, I’ll be putting together a simple RSS aggregator. It will print to the screen and write an html file with the channel titles and article titles.
Files Used in this Project
- RssReader.scala: The sourcecode for this project.
- feeds.txt: A sample input file.
Libraries
import scala.xml._
import java.net.URL
- scala.xml: Scala’s XML library has a vast number of features which I’ve only begun to sample.
- java.net.URL: Java’s library for URL’s and HTTP connections.
I’ve mentioned that Java libraries can be called as though they were Scala libraries elsewhere, this is the first time I’ve actually done it. It’s painless as advertised.
Main Functions, Arrays and Command Line Arguments
object RssReader {
def main(args : Array[String]) : Unit = {
val rssFeeds = combineFeeds(getRssFeeds(args(0)))
printFeeds(rssFeeds)
writeFeeds(rssFeeds)
}
One piece of syntax that is especially confusing in Scala is how it indicates that one type contains another, in this case Array[String]
. This syntax looks a lot like what you’ll see for array indexing in most other languages. In Scala arrays are treated as functions that receive an integer argument to return their values, eg args(0)
returns the first argument of args
. This is something of a simplification, but it helps me remember what the syntax is.
Anonymous Functions, Maps and Folds
def getRssFeeds(fileName : String) : List[(String, Seq[String])] = {
// Given a config file containing a list of rss feed URL's,
// returns a list with ('channel name', ['article 1', 'article 2'])
var baseList : List[(String, Seq[String])] = List()
getFileLines(fileName).map(extractRss).foldLeft(baseList)
{ (l, r) => l ::: r.toList }
}
Anonymous Functions
The function on line 4, (l, r) => l ::: r.toList
is a case of an anonymous function in Scala, I’ll break it down:
1. (l, r) =>
: The arguments section, if I wanted to be pedantic I could specify their type, but anonymous functions already clutter code and their purpose should usually be pretty obvious.
2. l ::: r.toList
: l
is an already existing list object which is being prepended to r
after r
is turned into a list.
Higher-order functions
Scala’s for
syntax is extraordinarily powerful, but sometimes it’s more than you need. The three most common higher-order functions are:
- map: Apply a function to each element in a sequence.
- filter: Use a boolean function to filter the elements of a sequence.
- fold(generally split into foldLeft and foldRight): Use a 2-arity function to combine all of the elements in a sequence into 1 element.
The for
syntax combines the functionality of map and filter, so is most useful when you need to do both to a sequence. It does not cover the functionality of fold. Here I’m:
1. Getting a sequence which contains the URL’s of several RSS feeds.
2. Mapping a extractRss
over them to get the RSS data.
3. Folding the anonymous function (l, r) => l ::: r.toList
over them to change them from immutable Seq values to Lists and combine them.
Filter and Some Sugar
def combineFeeds(feeds : List[(String, Seq[String])])
: List[(String, Seq[String])] = {
// Cleanup function, given a list of feeds it combines duplicate channels.
def combinedFeed(feedName : String) : Seq[String] = {
feeds.filter(_._1 == feedName).map(_._2).flatten
}
val feedNames = feeds.map(_._1).distinct
feedNames.map(x => (x, combinedFeed(x)))
}
The filter function in combineFeed
makes use of the fact that it is in combineFeeds
‘s namespace and doesn’t need to have feeds
passed to it. It filters through feeds
, keeping those with the same name. It then passes the result to map
which extracts the article lists, and flatten combines each list. Generally you would either pass a named function with a single argument to filter and map, or an anonymous function with the format (x) => x == feedName
. This pattern comes up so often that Scala allows us to ignore the usual anonymous function syntax and use the underscore as a placeholder for a single variable.
Brackets are Optional in Single Liners
def getFileLines(fileName : String) : Array[String] =
// Extracts the URL's from a file and breaks them up into individual strings.
scala.io.Source.fromFile(fileName).mkString.split("\n")
Opening Network Connections and Reading XML
def extractRss(urlStr : String) : Seq[(String, Seq[String])] = {
// Given a URL, returns a Seq of RSS data in the form:
// ("channel", ["article 1", "article 2"])
val url = new URL(urlStr)
val conn = url.openConnection
val xml = XML.load(conn.getInputStream)
for (channel < - xml \\ "channel")
yield {
val channelTitle = (channel \ "title").text
val titles = extractRssTitles(channel)
(channelTitle, titles)
}
}
Opening Network Connections
For practical purposes Lines 2-4 are straight Java. They format a URL, open a connection, download the stream and convert it into the XML library's internal format.
Reading the XML
Line 6 is where I start making good on my promise of easy XML processing. xml \\ "channel"
descends through the XML tree and extracts every element named "channel", we then use the for expression to take each of those elements and treat them each as XML trees in their own right. The \\
will descend through the entire tree, while \
only examines those nodes that are immediately under the given root.
def extractRssTitles(channel : Node) : Seq[String] = {
// Given a channel node from an RSS feed, returns all of the article names
for (title < - (channel \\ "item") \\ "title") yield title.text
}
In extractRssTitles
you can see how \\
can be nested to do more complex dives into an XML document.
Display
To Screen
def printFeeds(feeds : List[(String, Seq[String])]) : List[Seq[Unit]] = {
// Given a list of ("channel", ["article 1", "article 2"]), prints
// them each to screen.
for (feed < - feeds) yield {
println("*** " + feed._1 + " ***")
for (title <- feed._2) yield println("\t" + title)
}
}
printFeeds
takes a list of the feed objects and displays them on screen. There are a couple of ways to extract data from a tuple. If you're dealing with a complex set of structures, you can use match/case. I knew I'd always be dealing with a 2-tuple, so I chose to use the _#
syntax, where _1
will give you the first value in the tuple, etc.
To HTML
def writeFeeds(feeds : List[(String, Seq[String])]) = {
// Given a list of ("channel", ["article 1", "article 2"]), generates
// and writes an HTML document listing the articles with channel names
// as headers.
val html =
{for (feed < - feeds) yield
{feed._1}
{for (title < - feed._2) yield - {title}}
}
XML.saveFull("rss.html", html, "UTF-8", true, null)
}
}
When you produce XML in most languages you have to choose between using hard to read code that produces guaranteed well formed XML, or generating your XML out of strings that are easy to read but you can't be sure is well formed.
This is what has had me excessively excited about Scala for the past week. It seems that the majority of projects I get involved in sooner or later involve doing some kind of work with XML, and it's usually one of the ugliest parts of the job. Here you can see I have XML freely interspersed with the rest of my code, and was by far the easies part of what I wrote this week. At any time inside the XML, if I need to break back out into Scala I just surround the statement I want to use in curly braces. With it I've been able to clearly produce XML that I can be confident is well formed.
My Experience
Arrays
An unpleasant surprise I got while I was working on this program was that, unlike other sequence types, Arrays in Scala have a simple way to convert to Lists.
Functional Programming
Reading over my own code, I realize that functional style has become a habit. It'll be interesting to see how well I cope when I try to pick up a language where functional style isn't a realistic option. To anyone reading, if there are places in my Scala code where it would be more clear if I used a more object oriented style, please let me know.
XML
One more time. The XML processing capabilities of this language are beautiful. Scala is going to end up being part of my standard arsenal for this reason alone.