GSoC Status Update, week 5

japhy dnia 01 lipca 2008, 00:08:32 (CL-OpenID English Lisp Maszyny Techblog ).

Long time, no update; didn't write up for more than a week. My fault. The new code is still a mess, and every day I feel RP is just over the corner, and the next day, when it's done, I'll send the update. Doesn't seem to work this way. Probably in a few days at most, RP will be done; here's how it looks now:

Work done since last report

  • Fixed minor bugs and annoyances: typo in ASDF:TEST-OP definition (TEST-OP didn't work at all before), shadowing import of CL:NULL (for unknown reason, Ironclad shadows NULL symbol), corner cases in random testing, EVAL-WHEN around DEFCONSTANTS, some typos
  • Separated associations from ID alists. Association is not a part of an identity or of a verification process, as I first thought; it's basically a shared secret between Relying Party and an OpenID Provider. So, it had to be separated from ID flow and is now stored separately in its own structure. Information related to associations (valid association/session type lists) were also dropped from ID alists;
  • Implemented generation and checking of message signatures,
  • Implemented encoding alists to key-value form,
  • Indirect request support, authentication request, indirect reply handling (NFY), some (not all) checks for positive assertion verification; no tests yet for these; not fully finished yet, but commited anyway;
  • (uncommited) Hunchentoot handlers. They basically work, but are a bit of mess. I have the half-working Relying Party on my local development machine. After finishing the RP functionality, I can give the address to adventurous testers in private, if someone is interested in helping out.

Problems

Biggest problem was need to separate associations from IDs. It required quite a bit of refactoring and a change in my thinking about information flow within the protocol. Also, actual RP flow is a bit more tricky than I expected, and cleanly ftting protocol into HT while sustaining portability and separation between HTTP handlers anr protocol implementation isn't obvious.

Plans for next half week

  • Finish the Relying Party. At last. The milestone is already a week late;
  • Clean up the code: general bits, such as message formats or utility functions, not directly related to protocol flow, need to be put in a separate file instead of sticked near the first usage; also, there is a big need for docstrings (I want to write those later, since the internal API is quite a moving target at the moment);
  • Tests. Most of work done since last report has no unit tests;
  • External API.

  • After finishing prototype RP, I will provide a real data structure (class or a struct) for the ID; maybe some more refactoring will be needed. After that (possibly in parallel with refactoring), I'll start the OpenID Provider part. This may be a bit easier than RP, since I understand the protocol much better than when I started the RP, and much common code is already written. The part when actual authentication takes place will be challenging – it will need to be pluggable and support whatever authentication scheme the library user imagines.

Dodaj komentarz »

GSoC status update, week 4

japhy dnia 21 czerwca 2008, 23:14:59 (CL-OpenID English Lisp Maszyny Techblog ).

Work done since last report

  • Normalization of discovered information, add protocol version info to ID alist
  • Set valid session and association types on discovery
  • Communication formats: btwoc, key-value format, with tests
  • New dependency: trivial-utf-8, for portable and fast encoding strings. Needs a changeset not merged yet into mainline, which adds support for :START and :END parameters for UTF-8-BYTES-TO-STRING, which I wrote to reduce consing in key-value format message decoding
  • Association support, unencrypted and Diffie-Hellman, 1.1-compatible

Problems

Many of code that I wrote is not testable enough. There are tests for library functions (data format etc), but main information flow is still untested. I will need to refactor the code to improve testability. This may be in line with portability work – along with true implementations of HTTP client/server stuff, dummy implementation for unit testing the logic may be provided. Also, some refactoring of biggest functions – DISCOVER and ASSOCIATE – to split it into separate, unit-testable functions might be a Good Thing.

Plans for next half week

Finishing the non-portable client:

  • Signature generation and checking for protocol messages
  • Authentication request, authentication response verification
  • HTTP-server-side flow with Hunchentoot, for indirect messages and user-visible checks (if time permits)

Dodaj komentarz »

GSoC status update, week 3

japhy dnia 15 czerwca 2008, 23:47:08 (CL-OpenID English Lisp Maszyny Techblog ).

Whew! After passing all the exams, got around to doing XRDS parsing. This turned out to be a bit complex, and now only basic XRDS support is done. What is done, however, should cover most cases. Now on to real work. I'm a bit behind the schedule already, and full Relying Party is due next week – maybe I'll need to adjust the milestones, but I still hope I can do it.

Work done since last report

  • Minor fixes to HTML discovery, better discovery workflow
  • Fixes for compatibility with Clozure CL. Clozure works really well on x86_64 Linux, and probably I'll stick to it as a default development platform (keeping testing with other implementations) – way lighter and faster than SBCL.
  • Finished discovery of XRDS document location
  • XRDS parsing

Problems

As always, things take more time than I expect. This time, I tripped over underspecified XRDS document schema in the Yadis protocol; I noticed missing parts during implementation, and had to refer to XRI Resolution specification and to examples on the Internet. What's more, OpenID 1.x support in XRDS is not specified at all, there are only some conventions and “widely used” <XRD:Type/> URIs. Seems to work on real world providers.

Plans for next half-week

  • Association support

Dodaj komentarz »

GSoC status update, week 2

japhy dnia 07 czerwca 2008, 23:12:49 (CL-OpenID English Lisp Techblog ).

Discovery is almost finished now. HTML-based needs only tests written, and for Yadis, only XRDS analysis is left (plus tests, of course). For HTML parsing I use CL-HTML-Parse – it's light, doesn't drag any dependencies (only Allegroism it needs is the IF* macro, which is included), and handles HTML tag soup quite well.

Although I'm writing fairly separate bits of functionality, the need for some actual code architecture starts to show up. For every identity, the relying party gathers bits of information during various stages of protocol (and there are already two stages almost done), so I need to choose representation for this state. For now, to stay flexible while I need it, I made functions pass an alist with the information – initial alist is accepted by each function, and updated (possibly destructively) alist is returned. This is enough for now, and when (if) the need occurs, this shouldn't be hard to convert to proper object- or structure-based design.

Work done since last report

  • Researched and choose HTML parsing library
  • Researched public identity providers for reference identities
  • Implemented HTML discovery

Problems

Almost every public provider provides OpenID version 1. This makes compatibility with v1 much more important than I predicted, especially in the relying party. Also, many of those actually use Yadis, so its priority goes up. Yadis protocol, however, is really simple and I have most of it already done (no actual XRDS parsing yet, but this format is fairly simple and shouldn't take long).

Plans for next half week

  • Tests for HTML-based discovery and its helper functions (possible refactoring for better testability)
  • XRDS parsing & tests

Because of exams (two last finals on Monday and Wednesday, wish me luck – especially on Wednesday), I'll probably send next report on Wednesday. After it, back to regular Tuesday/Friday schedule.

1 komentarz »

Preambuła LaTeX-a

japhy dnia 07 czerwca 2008, 17:52:46 (Maszyny Techblog ).

Przeniesione ze strony domowej w ramach porządków. Preambuła, której skutecznie używam z pdfplatex z PLDowego teTeX-a. Który to teTeX, swoją drogą, jest już od ponad dwóch lat martwy i zaczyna nieładnie pachnieć. Dear Lazyweb, kiedy któryś z PLDowców (osobiście się boję) ugryzie TeX Live?

\documentclass[a4paper]{mwart}
\usepackage[utf8]{inputenc}
\usepackage{polski}
\usepackage[colorlinks]{hyperref}
\usepackage{graphicx}
\usepackage{listings}
\usepackage{ucs}

\renewcommand\abstractname{Streszczenie}

\begin{document}
\begin{titlepage}
  \author{Maciej Pasternacki}
  \title{}
  \date{\today}
  \maketitle
  \vfill
  \begin{abstract}

  \end{abstract}
  \setcounter{page}{0}          % to make hyperref not barf
  \thispagestyle{empty}
\end{titlepage}


\end{document}

% Local Variables:
% Mode: LaTeX
% TeX-PDF-mode: t
% LaTeX-command: "platex"
% End:

1 komentarz »

ParenQuery

japhy dnia 04 czerwca 2008, 19:27:06 (English Lisp Maszyny Techblog ).

;;; parenquery -- simple chaining JS method calls for JQuery

;;; instead of manual:
;; PARENQUERY> (ps (chain foo bar baz))
;; "foo.bar.baz;"
;; PARENQUERY> (ps (chain foo bar (baz)))
;; "foo.bar.baz();"
;; PARENQUERY> (ps (chain foo (bar 1 2 3) (baz)))
;; "foo.bar(1, 2, 3).baz();"
;; PARENQUERY> (ps (chain (me.next)
;;                        (show "slow" (lambda ()
;;                                       (chain ($ this) (filter "li") (css "display" "list-item"))
;;                                       (chain ($ this) (children) (eq 0) (focus))
;;                                       (chain ($ elt) (fade-out))
;;                                       (chain ($ this) (children) (filter ".__more") (fade-in))))))
;; "me.next().show('slow', function () {
;;     $(this).filter('li').css('display', 'list-item');
;;     $(this).children().eq(0).focus();
;;     $(elt).fadeOut();
;;     $(this).children().filter('.__more').fadeIn();
;; });"

(defpackage #:parenquery
  (:use #:common-lisp #:parenscript)
  (:export #:chain))
(in-package #:parenquery)

(defun %chain-1 (first second)
  (if (listp second)
      `((slot-value ,first ',(first second)) ,@(rest second))
      `(slot-value ,first ',second)))

(defun %chain (first second &rest rest)
  (if rest
      (apply #'%chain (%chain-1 first second) rest)
      (%chain-1 first second)))

(defpsmacro chain (first second &rest rest)
  "Chain methods, as for JQuery (jquery.org)"
  (apply #'%chain first second rest))

;; This program is free software. It comes without any warranty, to
;; the extent permitted by applicable law. You can redistribute it
;; and/or modify it under the terms of the Do What The Fuck You Want
;; To Public License, Version 2, as published by Sam Hocevar. See
;; http://sam.zoy.org/wtfpl/COPYING for details.

Dodaj komentarz »

GSoC status update, week 0.5

japhy dnia 02 czerwca 2008, 11:28:03 (CL-OpenID English Lisp Maszyny Techblog ).

Common-lisp.net's Trac site is down for last few days or so, which made the work a bit harder. Nevertheless, first real code has been written. Identifier normalization seems to work. I introduced another dependency, Puri – a portable version of Allegro's net.uri. Identifiers will be internally represented by Puri URI objects; for XRI, if I get to it (support for XRI is optional), I will probably subclass URI, so that minimal API changes will be needed.

I am pondering now how to do HTML parsing for HTML-based discovery. One way would be to use CL-HTML-Parse, or pxmlutils – a port of Allegro's xmlutils. Both would require adding dependencies to the library, dragging along (in case of pxmlutils) ACL-compat Allegro portability layer. Other way would be to add HTML parsing to XMLS. This may, however, seem tricky to do right, because HTML is not as structured as XML, and there may be many corner cases (possibly unquoted tag attributes, optional close tags, and so on). I don't know at the moment, how other parsers handle misformed HTML, and HTML's corner cases. If there is no reliable parser, an interface to a tool like HTML Tidy will be needed – either FFI, or by running external program

Work done this half-week

Problems

Minor only: reviewing available servers and installing one took more time than I expected, because much of information I needed to choose (like supported OpenID version, or required libraries) was well hidden inside the software. I also spent a few hours on what turned to be a documentation error in SimpleID during its installation.

Plans for next half-week

  • Research and choose HTML parsing strategy
  • Implement HTML-based discovery
  • Write tests for discovery and HTML parsing

Dodaj komentarz »

GSoC status update, week 0

japhy dnia 27 maja 2008, 23:41:03 (CL-OpenID English Lisp Maszyny Techblog ).

Your hacking starts... NOW! Official coding period has started yesterday. Dependencies have been downloaded, first lines of code have been written, first commit has just been pushed to the repository. The real work has started; now it's time to write actual code. ;) From now on, updates will be posted twice a week, every Tuesday and Friday.

Work done this week

  • Initialized code repository
  • Did research and initial choices on libraries:
    • Hunchentoot and Drakma for initial HTTP server and client
    • Ironclad for cryptography-related stuff
    • FiveAM for unit tests
    • For XML, no final decision yet… I'll start with XMLS for its simplicity, and maybe move on if it doesn't work well
    • Probably will need something for safe random number generation, will leave that one for later and stick with (RANDOM), seeded with /dev/random if possible, for now
    • Possibly some library for parsing will be needed; CL-PPCRE, or maybe some variant of Meta?
  • Linked and described repository, needed libraries, darcsweb and cut-and-paste HOWTO for getting the code with dependencies on Wiki front page
  • Prepared workplace, without local OpenID Provider for testing yet

Problems

Small delays related with final exams. Didn't complete list of existing implementations and reference providers, neither read much about the XRI thing. Still have some exams this week, and at least two more on June 9 and 11, but they should not cause significant delays

Plans for next half-week

  • Finish list of implementation and providers
  • Set up local OpenID Provider on my workstation for testing
  • Initial work on ID normalization plus unit tests

Lemonodor-fame is but a hack away!

2 komentarze »

GSoC status update, week -1

japhy dnia 21 maja 2008, 22:09:34 (CL-OpenID English Lisp Maszyny Techblog ).

This week I spent mostly fighting with infrastructure (Trac, web page, mailing lists, and so on), and reading further into specs. Just a week of research, and coding start date, May 26th, should be just right to actually start coding.

Today, I received a surprise book from Google. All the rumours on students' mailing list about it being beautiful are fully confirmed ;]

Work done this week

  • Further work on infrastructure: Trac versions, milestones, ticket types etc
  • Establishing rough roadmap, published at Roadmap page of project's Trac.
  • Completed referenes on ProtocolInformation Wiki page
  • Started a list of existing implementations and providers on ExistingImplementations Wiki page
  • More documentation reading: finished OpenID and Yadis; XRI and relevant RFCs are still left to read

Plans for next week

  • Initialize Darcs repository with code skeleton (directory structure, .asd file, package, etc), link darcsweb from main page and wiki
  • Finish list of existing implementation and providers
  • Complete list of libraries that I will use (especially cryptography and XML-related ones)

Dodaj komentarz »

GSoC status update, week -2

japhy dnia 13 maja 2008, 21:59:49 (CL-OpenID English Lisp Maszyny Techblog ).

Dla polskich czytelników: od teraz do września na blogu publikował będę raz lub dwa razy w tygodniu status swojego projektu w Summer of Code 2008 (piszę lispową implementację OpenID, strona projektu jest pod adresem http://common-lisp.net/project/cl-openid/). Jest to jeden z oficjalnych kanałów komunikacji związanej z projektem, więc wiadomości pojawiać się będą w języku angielskim.

This year, I got accepted to Google Summer of Code programme. I will work on Common Lisp implementation of OpenID protocol; project page is at http://common-lisp.net/project/cl-openid/, and full proposal is available on project's Trac Wiki at OriginalProposal page. I will publish status updates twice a week (weekly before coding, which starts May 26) on this blog, in category CL-OpenID (RSS). This is the first update, two weeks before coding start.

Work done this far

  • Discussed project organization with mentor, Anton Vodonosov;
  • Chosen the license (LLGPL), version control method (darcs), and hosting (common-lisp.net);
  • Created common-lisp.net project, with mailing lists and Trac Wiki and issue tracker;
  • Initial overall design, described on InitialDesign Wiki page;
  • Initial reading of the specification.

Plans for next week

  • Further reading into specifications, especially RFCs referred to by OpenID spec;
  • Estimating rough project time line;
  • Research on existing OpenID implementations for comparison, ideas and testing.

Dodaj komentarz »