GSoC Status Update, week 7½

japhy dnia 15 lipca 2008, 23:43:02 (CL-OpenID English Lisp Maszyny Techblog ).

This week, first potential user showed up on the devel list, and even sent the bug report (for trivial-utf-8, actually, and it's already fixed upstream). Wow! It seems that my stuff may be actually useful. ;)

Work done since last report

  • Refactored association.lisp for more general association object construction and Diffie-Hellman negotiation
  • Sorted out a bit general Provider skeleton design
  • Implemented association support for OpenID Provider (still polishing, not commited yet)

Problems

Bits appearing where they should not. More specifically, I implemented the association support, but there are still bugs: in Diffie-Hellman key exchange (only in provider part, even though function for DH calculations is shared between OP and RP), the public value sent by provider is (usually, but not always) 129 bits long, instead of 128, with no apparent reason. Must be some mistake on my part in the modular arithmetic; I still have to hunt it down, and then the OP's associations are commitable.

Plans for next half week

I may not be able to give substantial time to project before Sunday – my girlfriend is moving out for her PhD, and I will need to help with packing and carrying boxes (and the house is in such a mess right now that it's getting hard to concentrate – and it's just the beginning of packing…). However, I hope I can at least handle the DH bug. I will make up for this from Sunday on. Summarizing, I plan to:

  • Hunt down the Diffie-Hellman bug, commit OP associations
  • Possibly some later parts of OP's conversations – authentication request, signatures – if time and circumstances permit.

1 komentarz »

GSoC Status Update, week 7

japhy dnia 11 lipca 2008, 14:49:03 (CL-OpenID English Lisp Maszyny Techblog ).

Work done since last report

  • Minor bugfixes in RP protocol
  • Started work on OpenID Provider non-portable prototype (rough sketches, not commited yet).
  • Choose reference relying party for initial tests: I use JanRain's python-openid, which I run directly from IPython

Problems

This time, none.

Plans for next half week

Work on OP prototype.
  • Some frame for an endpoint
  • Association support
  • If tme permits, maybe rough authentication

Dodaj komentarz »

GSoC Status Update, week 6

japhy dnia 07 lipca 2008, 18:27:08 (CL-OpenID English Lisp Maszyny Techblog ).

Finally, the non-portable Relying Party is here. There is still a bit of duct tape here and there, and it cries for refactoring. The ID token deserves its own structure or class, and instead of global variables there should be a first-class Relying Party object with a bit of CLOS protocol and maybe callbacks (or maybe just inheritance and specialization will be enough). It may cry, but refactoring will have to wait; first, I wish to have non-portable OpenID Provider (or OP) prototype, so I can figure out common parts of the code and understand the protocol better.

Not all protocol options are explicitly supported yet: at least immediate check does not fit in the flow. This will be fixed with the refactoring and the first-class RP. There are some bugs left for sure, and unit tests aren't there yet – they'd need to be replaced during refactoring ayway.

Work done since last report

  • Support for openid.invalidate_handle reply field and checking signatures directly with OP
  • OpenID 1.x compatibility in RP
  • Polished and commited whole Relying Party flow and Hunchentoot handlers
  • Polished Authentication Response handling

Problems

Biggest and, I suppose, only problem since last report (or even two) was hitting the Ninety/Ten Rule: in last two weeks or so I did the last 10% of work, which took the other 90% of time. Debugging, corner cases, polishing… there is still some of these left to do, I think, but a prototype is working, and I'll take care of the details when I will do The Refactoring.

Plans for next half week

Start the non-portable prototype OpenID Provider. I won't try to precisely estimate how much would the provider prototype take; I think the safe bet would be two to three weeks.

Dodaj komentarz »

GSoC Status Update, week 5

japhy dnia 01 lipca 2008, 00:08:32 (CL-OpenID English Lisp Maszyny Techblog ).

Long time, no update; didn't write up for more than a week. My fault. The new code is still a mess, and every day I feel RP is just over the corner, and the next day, when it's done, I'll send the update. Doesn't seem to work this way. Probably in a few days at most, RP will be done; here's how it looks now:

Work done since last report

  • Fixed minor bugs and annoyances: typo in ASDF:TEST-OP definition (TEST-OP didn't work at all before), shadowing import of CL:NULL (for unknown reason, Ironclad shadows NULL symbol), corner cases in random testing, EVAL-WHEN around DEFCONSTANTS, some typos
  • Separated associations from ID alists. Association is not a part of an identity or of a verification process, as I first thought; it's basically a shared secret between Relying Party and an OpenID Provider. So, it had to be separated from ID flow and is now stored separately in its own structure. Information related to associations (valid association/session type lists) were also dropped from ID alists;
  • Implemented generation and checking of message signatures,
  • Implemented encoding alists to key-value form,
  • Indirect request support, authentication request, indirect reply handling (NFY), some (not all) checks for positive assertion verification; no tests yet for these; not fully finished yet, but commited anyway;
  • (uncommited) Hunchentoot handlers. They basically work, but are a bit of mess. I have the half-working Relying Party on my local development machine. After finishing the RP functionality, I can give the address to adventurous testers in private, if someone is interested in helping out.

Problems

Biggest problem was need to separate associations from IDs. It required quite a bit of refactoring and a change in my thinking about information flow within the protocol. Also, actual RP flow is a bit more tricky than I expected, and cleanly ftting protocol into HT while sustaining portability and separation between HTTP handlers anr protocol implementation isn't obvious.

Plans for next half week

  • Finish the Relying Party. At last. The milestone is already a week late;
  • Clean up the code: general bits, such as message formats or utility functions, not directly related to protocol flow, need to be put in a separate file instead of sticked near the first usage; also, there is a big need for docstrings (I want to write those later, since the internal API is quite a moving target at the moment);
  • Tests. Most of work done since last report has no unit tests;
  • External API.

  • After finishing prototype RP, I will provide a real data structure (class or a struct) for the ID; maybe some more refactoring will be needed. After that (possibly in parallel with refactoring), I'll start the OpenID Provider part. This may be a bit easier than RP, since I understand the protocol much better than when I started the RP, and much common code is already written. The part when actual authentication takes place will be challenging – it will need to be pluggable and support whatever authentication scheme the library user imagines.

Dodaj komentarz »

GSoC status update, week 4

japhy dnia 21 czerwca 2008, 23:14:59 (CL-OpenID English Lisp Maszyny Techblog ).

Work done since last report

  • Normalization of discovered information, add protocol version info to ID alist
  • Set valid session and association types on discovery
  • Communication formats: btwoc, key-value format, with tests
  • New dependency: trivial-utf-8, for portable and fast encoding strings. Needs a changeset not merged yet into mainline, which adds support for :START and :END parameters for UTF-8-BYTES-TO-STRING, which I wrote to reduce consing in key-value format message decoding
  • Association support, unencrypted and Diffie-Hellman, 1.1-compatible

Problems

Many of code that I wrote is not testable enough. There are tests for library functions (data format etc), but main information flow is still untested. I will need to refactor the code to improve testability. This may be in line with portability work – along with true implementations of HTTP client/server stuff, dummy implementation for unit testing the logic may be provided. Also, some refactoring of biggest functions – DISCOVER and ASSOCIATE – to split it into separate, unit-testable functions might be a Good Thing.

Plans for next half week

Finishing the non-portable client:

  • Signature generation and checking for protocol messages
  • Authentication request, authentication response verification
  • HTTP-server-side flow with Hunchentoot, for indirect messages and user-visible checks (if time permits)

Dodaj komentarz »

GSoC status update, week 3

japhy dnia 15 czerwca 2008, 23:47:08 (CL-OpenID English Lisp Maszyny Techblog ).

Whew! After passing all the exams, got around to doing XRDS parsing. This turned out to be a bit complex, and now only basic XRDS support is done. What is done, however, should cover most cases. Now on to real work. I'm a bit behind the schedule already, and full Relying Party is due next week – maybe I'll need to adjust the milestones, but I still hope I can do it.

Work done since last report

  • Minor fixes to HTML discovery, better discovery workflow
  • Fixes for compatibility with Clozure CL. Clozure works really well on x86_64 Linux, and probably I'll stick to it as a default development platform (keeping testing with other implementations) – way lighter and faster than SBCL.
  • Finished discovery of XRDS document location
  • XRDS parsing

Problems

As always, things take more time than I expect. This time, I tripped over underspecified XRDS document schema in the Yadis protocol; I noticed missing parts during implementation, and had to refer to XRI Resolution specification and to examples on the Internet. What's more, OpenID 1.x support in XRDS is not specified at all, there are only some conventions and “widely used” <XRD:Type/> URIs. Seems to work on real world providers.

Plans for next half-week

  • Association support

Dodaj komentarz »

GSoC status update, week 2

japhy dnia 07 czerwca 2008, 23:12:49 (CL-OpenID English Lisp Techblog ).

Discovery is almost finished now. HTML-based needs only tests written, and for Yadis, only XRDS analysis is left (plus tests, of course). For HTML parsing I use CL-HTML-Parse – it's light, doesn't drag any dependencies (only Allegroism it needs is the IF* macro, which is included), and handles HTML tag soup quite well.

Although I'm writing fairly separate bits of functionality, the need for some actual code architecture starts to show up. For every identity, the relying party gathers bits of information during various stages of protocol (and there are already two stages almost done), so I need to choose representation for this state. For now, to stay flexible while I need it, I made functions pass an alist with the information – initial alist is accepted by each function, and updated (possibly destructively) alist is returned. This is enough for now, and when (if) the need occurs, this shouldn't be hard to convert to proper object- or structure-based design.

Work done since last report

  • Researched and choose HTML parsing library
  • Researched public identity providers for reference identities
  • Implemented HTML discovery

Problems

Almost every public provider provides OpenID version 1. This makes compatibility with v1 much more important than I predicted, especially in the relying party. Also, many of those actually use Yadis, so its priority goes up. Yadis protocol, however, is really simple and I have most of it already done (no actual XRDS parsing yet, but this format is fairly simple and shouldn't take long).

Plans for next half week

  • Tests for HTML-based discovery and its helper functions (possible refactoring for better testability)
  • XRDS parsing & tests

Because of exams (two last finals on Monday and Wednesday, wish me luck – especially on Wednesday), I'll probably send next report on Wednesday. After it, back to regular Tuesday/Friday schedule.

1 komentarz »

ParenQuery

japhy dnia 04 czerwca 2008, 19:27:06 (English Lisp Maszyny Techblog ).

;;; parenquery -- simple chaining JS method calls for JQuery

;;; instead of manual:
;; PARENQUERY> (ps (chain foo bar baz))
;; "foo.bar.baz;"
;; PARENQUERY> (ps (chain foo bar (baz)))
;; "foo.bar.baz();"
;; PARENQUERY> (ps (chain foo (bar 1 2 3) (baz)))
;; "foo.bar(1, 2, 3).baz();"
;; PARENQUERY> (ps (chain (me.next)
;;                        (show "slow" (lambda ()
;;                                       (chain ($ this) (filter "li") (css "display" "list-item"))
;;                                       (chain ($ this) (children) (eq 0) (focus))
;;                                       (chain ($ elt) (fade-out))
;;                                       (chain ($ this) (children) (filter ".__more") (fade-in))))))
;; "me.next().show('slow', function () {
;;     $(this).filter('li').css('display', 'list-item');
;;     $(this).children().eq(0).focus();
;;     $(elt).fadeOut();
;;     $(this).children().filter('.__more').fadeIn();
;; });"

(defpackage #:parenquery
  (:use #:common-lisp #:parenscript)
  (:export #:chain))
(in-package #:parenquery)

(defun %chain-1 (first second)
  (if (listp second)
      `((slot-value ,first ',(first second)) ,@(rest second))
      `(slot-value ,first ',second)))

(defun %chain (first second &rest rest)
  (if rest
      (apply #'%chain (%chain-1 first second) rest)
      (%chain-1 first second)))

(defpsmacro chain (first second &rest rest)
  "Chain methods, as for JQuery (jquery.org)"
  (apply #'%chain first second rest))

;; This program is free software. It comes without any warranty, to
;; the extent permitted by applicable law. You can redistribute it
;; and/or modify it under the terms of the Do What The Fuck You Want
;; To Public License, Version 2, as published by Sam Hocevar. See
;; http://sam.zoy.org/wtfpl/COPYING for details.

Dodaj komentarz »

GSoC status update, week 0.5

japhy dnia 02 czerwca 2008, 11:28:03 (CL-OpenID English Lisp Maszyny Techblog ).

Common-lisp.net's Trac site is down for last few days or so, which made the work a bit harder. Nevertheless, first real code has been written. Identifier normalization seems to work. I introduced another dependency, Puri – a portable version of Allegro's net.uri. Identifiers will be internally represented by Puri URI objects; for XRI, if I get to it (support for XRI is optional), I will probably subclass URI, so that minimal API changes will be needed.

I am pondering now how to do HTML parsing for HTML-based discovery. One way would be to use CL-HTML-Parse, or pxmlutils – a port of Allegro's xmlutils. Both would require adding dependencies to the library, dragging along (in case of pxmlutils) ACL-compat Allegro portability layer. Other way would be to add HTML parsing to XMLS. This may, however, seem tricky to do right, because HTML is not as structured as XML, and there may be many corner cases (possibly unquoted tag attributes, optional close tags, and so on). I don't know at the moment, how other parsers handle misformed HTML, and HTML's corner cases. If there is no reliable parser, an interface to a tool like HTML Tidy will be needed – either FFI, or by running external program

Work done this half-week

Problems

Minor only: reviewing available servers and installing one took more time than I expected, because much of information I needed to choose (like supported OpenID version, or required libraries) was well hidden inside the software. I also spent a few hours on what turned to be a documentation error in SimpleID during its installation.

Plans for next half-week

  • Research and choose HTML parsing strategy
  • Implement HTML-based discovery
  • Write tests for discovery and HTML parsing

Dodaj komentarz »

GSoC status update, week 0

japhy dnia 27 maja 2008, 23:41:03 (CL-OpenID English Lisp Maszyny Techblog ).

Your hacking starts... NOW! Official coding period has started yesterday. Dependencies have been downloaded, first lines of code have been written, first commit has just been pushed to the repository. The real work has started; now it's time to write actual code. ;) From now on, updates will be posted twice a week, every Tuesday and Friday.

Work done this week

  • Initialized code repository
  • Did research and initial choices on libraries:
    • Hunchentoot and Drakma for initial HTTP server and client
    • Ironclad for cryptography-related stuff
    • FiveAM for unit tests
    • For XML, no final decision yet… I'll start with XMLS for its simplicity, and maybe move on if it doesn't work well
    • Probably will need something for safe random number generation, will leave that one for later and stick with (RANDOM), seeded with /dev/random if possible, for now
    • Possibly some library for parsing will be needed; CL-PPCRE, or maybe some variant of Meta?
  • Linked and described repository, needed libraries, darcsweb and cut-and-paste HOWTO for getting the code with dependencies on Wiki front page
  • Prepared workplace, without local OpenID Provider for testing yet

Problems

Small delays related with final exams. Didn't complete list of existing implementations and reference providers, neither read much about the XRI thing. Still have some exams this week, and at least two more on June 9 and 11, but they should not cause significant delays

Plans for next half-week

  • Finish list of implementation and providers
  • Set up local OpenID Provider on my workstation for testing
  • Initial work on ID normalization plus unit tests

Lemonodor-fame is but a hack away!

2 komentarze »