The first alpha version of OCaml Batteries Included has landed. It is now available as source code, from the Forge or as a GODI package. You may find the API documentation here and the complete documentation there.

Remember, this is alpha-level code, use at your own risk. Also note that documentation generation is very slow (10+ minute on my laptop), so don't worry if installation seems to last forever, it's nearly true but not quite.

Reviews, comments, suggestions and bug reports are particularly welcome. We have a few trackers for these, as well as a forum, so don't hesitate to use them. We're also looking for volunteers to give us a hand, so please consider stepping forward.

What is OCaml Batteries Included?

Twenty years ago, a language was just a compiler. You could measure the quality of a language from the beauty of its semantics, the clarity of its syntax, the speed of generated code. That was then. In the meantime, the Java and .Net nuclear plants have been built, while the Python and Ruby communities have gradually developed their own powerhouses. All these platforms have amply demonstrated that a language can only be as beautiful, clear and fast as the libraries which developers are actually going to use for their work. In other words, it's not about the language anymore, it's about the platform.

At this point in time, out-of-the-box, OCaml isn't a usable platform. There is no Unicode, there are no modern user interface toolkits, no distributed programming infrastructures, no network services, no type-safe communications, no analysis of other languages, no interfacing with industrial platforms, no XML, non modern two- or three-dimensional drawing engine, etc. This would not be too much of a problem if OCaml provided an easy way of installing new libraries and of using these libraries once they are installed. This would also not be too much of a problem if OCaml could somehow guarantee that trivial data structures didn't need to be reinvented by most projects and that communication between libraries happened lawlessly.

So no, out of the box, OCaml isn't a usable platform. However, no matter what you're trying to do, chances are that the community has already developed or adapted a tool to make your life easier. Easy installation of OCaml packages is quite possible, if you are using Debian or Ubuntu (apt-get), Fedora or Red Hat (yum) or actually, any Unix-compatible platform, including MacOS X and Windows (GODI). Simple and reliable usage of installed libraries can be done with Findlib. A comprehensive Unicode library is available (Camomile), as well as a modern user interface toolkit or two (LablGtk, OCamlRT), a number of distributed programming infrastructures (OCamlMPI, BSML, Opis, Camlp3l, OCamlNAE), libraries to interface with industrial platforms
(OCamlJava, SpiderCaml ...), to analyse other languages (CIL, Dalton/FlowCaml, ...), to read or write XML (PXP, Expat, ...), to draw in two or three dimensions (Cairo, OpenGL), to test your programs (OUnit), etc. OCaml even offers a built-in tool to customize the language itself (Camlp4).

What's missing? A few things. For now, not all important libraries are available as simple-to-install packages. That problem is being addressed by the devoted packagers of Debian, Fedora and GODI, while the possibility exists that the recently announced Symbiosis will also help address the issue. The other missing part is a standard set of libraries, language extensions and data structures which developers could be assured to find on every OCaml installation, and which would let them write programs without having to endlessly reinvent the same basic wheels, and without spending their time writing adapters between libraries which should work well together but don't use the same conventions. OCaml Batteries Included is one possible answer to this problem.

OCaml Batteries Included consists in

• A core set of libraries, designed to define the basic standard data structures. This set, largely based on both the Base library of OCaml and ExtLib, extends the basic strings, arrays, lists... provided with OCaml and introduces numerous data structures, including enumerations, lazy lists, extendable inputs
  and outputs, dynamic arrays, unicode ropes...
• A uniformization layer, the glue, on top of chosen existing libraries. Note that we are not forking any of these libraries, only providing an additional layer on top of them.  The libraries may be installed manually by the user or, preferably, by automatic dependency resolution thanks to apt-get, yum, GODI or some other packaging tool. The uniformization layer serves to guarantee that libraries play together nicely, that only one manner of reading from a file or writing to the output is necessary, that every data provided by a library may be decoded by another, etc.
• Additional documentation on whichever library for which we provide glue, including both low-level documentation ("what the heck does this function do?"), mid-level documentation ("why should I use that function?") and high-level documentation ("ok,I'm new here, where do I start?")
• A handful of language extensions, provided as Camlp4 modules, to solve common issues, automatically generate boilerplate code, improve readability, ...
• In the future, possibly a set of external tools.
• Build tools to make all of this transparent.
• And, of course, a logo.

[caption id="attachment_184" align="alignnone" width="439" caption="(Batteries courtesy of Benjamin Pavie, Camel courtesy of John Olsen, no animals injured) "][/caption]

OCaml Batteries Included is a project maintained by the community, which means that it depends on you. If you have ideas, suggestions, complaints, bug reports, if you want to participate, to write code, documentation, tutorials, build tools, review code, have a word in policies, if you want a package to be included in the Batteries, or just to contact us, please visit our website and take advantage of our bug trackers, request for features trackers, forums, etc. (courtesy of OCaml Core). We can also often be seen on irc, on server freenode, channel #ocaml and, of course on the OCaml Mailing-List.

OCaml Batteries Included is also a work-in-progress. The version you have in front of your eyes does not contain everything we want to put in it, nor even all the code we have written for it. But we'll get there. We intend to integrate additional libraries progressively, from minor release to minor release, with major milestones approximately twice per year. This policy is not written in stone and is largely subject to debate, so don't hesitate to comment on the subject.

For more informations on the contents of OCaml Batteries Included, follow us towards the manual.

Relations to other libraries

Project Gallium's Base Library

First, a word on vocabulary. We call "Base library" the library provided by INRIA with the default distribution of OCaml. We don't call it "standard library" for the simple reason that there are several libraries vying for the status of standard, including Batteries Included.

The relation between Batteries Included and the Base Library is simple: the Base Library is one of the libraries for which Batteries Included provides a uniformization layer. We are not forking the library, merely providing additional functions, additional documentation, boilerplate code...

The complete Base Library is available in Batteries Included as module Legacy. Most modules are also available inside the Batteries module hierarchy, sometimes under different names, and usually completed by numerous new functions. A few modules are considered obsolete and appear only in Legacy.

Jane Street's Core

Jane Street's Core is another library vying for the status of standard, this time produced by Jane Street Capital. This library is comparable in purpose and design to the core of Batteries Included (actually, we draw some inspiration from them and we hope that they are going to draw some inspiration from us, too) but there is no code shared between Batteries Included and Jane Street's Core for the moment.

In the future, Batteries Included may depend on Jane Street's Core. This is not the case yet as, according to one of Jane Street's Core's authors, this library may change quite a lot, and in what we understand as possibly incompatible ways, before reaching version 1.0.

We already depend on two components of Jane Street's Core: Type-Conv (a general infrastructure which may be used to generate boilerplate code) and Sexplib (an instance of Type-Conv used to generate code for serializing to/deserializing from human-readable S-Expressions). We intend to extend this to a third component of Jane Street's Core: Bin-Prot, another instance of Type-Conv, used this time to generate serialization to/from binary protocols.

ExtLib

ExtLib is another extension of the OCaml Base Library. ExtLib was designed as a relatively small addition, with the idea of fitting nicely within the OCaml Base Library. Since 2005, ExtLib has essentially stopped growing, following a conscious choice from the maintainers.

OCaml Batteries Included is largely based on ExtLib. Indeed, the core of Batteries is essentially a fork of ExtLib. We intend to maintain  ascending compatibility with ExtLib (with the exception of changes in module names), although we have added a great number of features,  fixed bugs, split code, etc. For most purposes, OCaml Batteries Included contains ExtLib and, in our mind, superseeds this library. We hope that the maintainers and developers of ExtLib will consider migrating to OCaml Batteries Included and helping us with their skills.

Community OCaml

Community OCaml is an attempt to turn OCaml into an out-of-the-box complete development platform, including numerous libraries.  We have joined forces with the developers of Community OCaml and we share most of our code. Although the design of Community OCaml is not completely decided yet, it is quite possible that this will end up as distribution consisting in OCaml + Batteries Included + some package manager.

OCamlNet

OCamlNet is an implementation of numerous Internet protocols for OCaml, along with so many utilities that it almost constitues a complete general-purpose development library for OCaml.

OCaml Batteries Included will depend on OCamlNet and incorporate OCamlNet into its hierarchy. In particular, for the moment, OCaml Batteries Included provides little in the way of libraries for interacting with the operating system, as we intend to use OCamlNet for that purpose.

Caml Development Kit

The Caml Development Kit, or CDK, was another attempt to build a development platform around OCaml by bundling together OCaml itself and a number of interesting libraries and tools. By opposition to Batteries Included, this was a monolithic distribution, with a custom compiler and very little documentation. Development on CDK seems to have died around 2001.

There is no relation between CDK and Batteries Included other than the common goal. Any feedback from former CDK users or developers will be welcome, though.

GODI, Apt, Yum

GODI, Apt and Yum are the three main package managers available for OCaml users. The first one is OCaml-specific, the second one lives in Debian/Ubuntu/Knoppix world and the third one in the Red Hat/Fedora world.

OCaml Batteries Included has no direct relation with either package manager, nor does it replicate the work of any of these managers. However, as Batteries relies on numerous external libraries, use of a package manager is strongly recommended.

Findlib

Findlib is a compile-time package manager for OCaml. It lets developers easily specify which libraries are required and manages compile-time dependencies between installed libraries.

OCaml Batteries Included uses Findlib extensively.

Comparable projects, somewhere else

Python Batteries Included

To the best of our knowledge, the term "Batteries Included" was first coined by the Python community to describe the standard distribution of Python, which in addition to usual data structures, contains modules for handling databases, compression, a number of file format (de)coders including JSON, (X)HTML, XML, CSV, cryptography, logging, text interfaces, threading, inter-process communication, implementation of client and server protocols from SMTP to HTTP, communication with external webbrowsers, image and sound manipulation, internationalization, lexing and parsing, graphical interfaces, unit testing, sandboxing, reflexivity, OS-specific services, ...

This massive number of out-of-the-box features, along with the conciseness of the language, are probably the two main reasons of the success of Python: simple tasks, even those which require complex libraries, may often be programmed with only a few lines and in a few minutes.

Well, obviously, we're trying to provide something as useful with OCaml Batteries Included, but with the added safety and speed of OCaml.

Haskell Batteries Included

The Haskell Platform (although known as "Haskell Batteries Included") is a recent project undertaken by the Haskell community with objectives comparable to OCaml Batteries Included: "provides a comprehensive,
stable and quality tested base for Haskell projects to work from." At the time of this writing, the Haskell Platform has not released any software, although a first release is expected within a few weeks.

As OCaml Batteries, the Haskell Platform is community-led and relies on a number of decentralized libraries. As OCaml Batteries, the Haskell Platform requires package management.There are a number of differences in the methods, though.

Some differences are probably trivial: where we strive to work with any of the major package management systems, the Haskell Platform is based solely on Cabal, which may allow them a better integration. Where OCaml Batteries attempts to reclassify existing libraries into one uniform hierarchy of modules, the Haskell Platform keeps the original module names, as decided by their original authors, pre-Platform.

Others are quite far-reaching: OCaml Batteries provides an extended core of libraries to serve as support for standardization and uniformization, while the Haskell Platform doesn't. Similarly, the Haskell Platform doesn't add any uniformization layer, which means that the task of getting libraries to work together lies upon the end-user. Some Haskell libraries may be patched into compliance and uniformization, but this is not always possible, short of creating cyclic dependencies between libraries which should work together but don't. Finally, the Haskell Platform has much stricter guidelines regarding libraries which can or can't make it into the Platform. This is a good thing if library authors are willing to fix whichever problems prevent the inclusion of their work -- something which we don't assume for OCaml Batteries, at least not yet.

Despite these differences, both projects seem based on solid foundations. And hopefully, both will achieve large success.

<Query>
  <Where>
    <Eq>
      <FieldRef Name="ID"/>
      <Value Type="Integer"><UserID Type="Integer"/></Value>
    </Eq>
  </Where>
</Query<

Just as a note, don't forget to put Type="Integer" in there.

If you wonder how I got over this, you should read my other article.

1. Go to your SharePoint
2. Create a list
3. Create a view for the list (for the sake of simplicity let's name it My View)
4. Choose the conditions that you're interested in for the query, let's say all the items modified today, and save the view.
   [caption id="attachment_118" align="aligncenter" width="300" caption="View filter in SharePoint"][/caption]
5. Go to the list settings and save the list as a template
   [caption id="attachment_120" align="aligncenter" width="300" caption="Save list as template in SharePoint"][/caption]
   [caption id="attachment_122" align="aligncenter" width="300" caption="List template is saved"][/caption]
6. Go to the template list library
   [caption id="attachment_128" align="aligncenter" width="300" caption="List template gallery"][/caption]
7. Download the template
8. The template is actually a zip file. Rename it from mylist.stp to mylist.zip and extract the manifest.xml file from it.
9. Now open the XML file (with FireFox) and go to the element /ListTemplate/UserLists/List/Metadata/Views/View[@DisplayName = 'My View'](the first /ListTemplate/UserLists/List/Metadata/Views/View for which the attribute DisplayName is My View). Under this element you will find a Query element. This is what you need, your CAML query.
   

And that manifest.xml file is also the source of some other useful stuff like the internal name of the fields (the StaticName attribute of the /ListTemplate/UserLists/List/Metadata/Fields/Field elements).

I was trying to write Caml Query to retrieve list items from Sharepoint, so I decided to write a post about that. First; Operators in Caml Queries are;

• Eq                           Equals
• Neq                        Not equal
• Gt                           Greater than
• Ge                           Greater than or equal
• Lt                            Lower than
• Le                           Lower than
• IsNull                    Is null
• BeginsWith         Begins with
• Contains              Contains

Here comes a simple query, which can get list items whose FieldA property is equal to ChoiceB;

<Where>

<Eq>

<FieldRef Name="FieldA" />

<Value Type="Choice">ValueB</Value>

</Eq>

 </Where>  

 You can add as many filters as you like by using And or Or. And lets compare Date values now;    

<Where>

                <And>

<Leq>

<FieldRef Name="FieldStartDate" />

<Value Type="DateTime" IncludeTimeValue="False">

<Today/>

</Value>

</Leq>

<Geq>

<FieldRef Name="FieldEndDate" />

<Value Type="DateTime" IncludeTimeValue="False">

<Today/>

</Value>

</Geq>

</And>

</Where>

 You can use <Today/> to give today’s date. Or you can use <Now/> if you want to also take time part account.

By the way, you can use something like <Today OffsetDays=”+3” /> if you want to say 3 days after today.

To get previous days use (-) (negative numbers), or to get next days use (+) (positive numbers)

I searched but could not find something like OffsetMonths, if you know, please let me know.

Also I should say, one of the common mistakes, I am always doing; writing more than 2 statements inside an “And” or “Or”. You should not write more than 2 statements!

Just write it like;

<Where>

                <And>

                <And>

<Leq>

…..

</Leq>

<Geq>

…..

</Geq>

</And>

<Geq>

…..

</Geq>

</And>

</Where>

And finally order statement should look like that;

<OrderBy>

<FieldRef Name="FieldA" Ascending="True"/>

</OrderBy>

Take a look at my final CAML query; (just as an example)

<Query>

<Where>

<And>

<Eq>

<FieldRef Name="FieldA" />

<Value Type="Choice">ChoiceA</Value>

</Eq>

<Or>

<Or>

<And>

<Geq>

<FieldRef Name="FieldStartDate" />

<Value Type="DateTime" IncludeTimeValue="False">

<Today/>

</Value>

</Geq>

<Leq>

<FieldRef Name="FieldStartDate" />

<Value Type="DateTime" IncludeTimeValue="False">

<Today OffsetDays="+10" />

</Value>

</Leq>

</And>

<And>

<Geq>

<FieldRef Name="FieldEndDate" />

<Value Type="DateTime" IncludeTimeValue="False">

<Today />

</Value>

</Geq>

<Leq>

<FieldRef Name="FieldEndDate" />

<Value Type="DateTime" IncludeTimeValue="False">

<Today OffsetDays="+10" />

</Value>

</Leq>

</And>

</Or>

<And>

<Leq>

<FieldRef Name="FieldStartDate" />

<Value Type="DateTime" IncludeTimeValue="False">

<Today />

</Value>

</Leq>

<Geq>

<FieldRef Name="FieldEndDate" />

<Value Type="DateTime" IncludeTimeValue="False">

<Today OffsetDays="+10" />

</Value>

</Geq>

</And>

</Or>

</And>

</Where>

<OrderBy>

<FieldRef Name="FieldStartDate" Ascending="True" />

</OrderBy>

</Query>

Found out that there is a CAML Builder tool, you can check that out from this site...

 <Field
    ID="{5B9985AC-C959-458d-BE78-A464A013C0C9}"
    Name="SecondaryBusinessSponsor"
    DisplayName="Secondary Business Sponsors"
    Type="UserMulti"  
    Group="TMG Columns"
  Mult="TRUE">
  </Field>

The lambda calculus provides the model for functional programming. Modern functional languages can be viewed as embellishments to the lambda calculus.

While not fully functional, both the original Lisp and APL were important in the development of functional programming. Later versions of Lisp such as Scheme and variants of APL did provide full functional support. Other important functional languages include Erlang, Haskell, and ML.

Functional programming languages, especially purely functional ones, have largely been emphasized in academia rather than in commercial software development. However, notable functional programming languages used in industry and commercial applications include Erlang, OCaml, Haskell, Scheme (since 1986) and domain-specific programming languages like R (statistics), Mathematica (symbolic math), J and K (financial analysis), and XSLT (XML).

What is a purely functional language

Functional programming languages are informally classified into pure and impure languages. The precise meaning of this distinction has been a matter of controversy. We therefore investigate a formal definition of purity. We begin by showing that some proposed definitions that rely on confluence, soundness of the beta axiom, preservation of pure observational equivalences, and independence of the order of evaluation, do not withstand close scrutiny. We propose instead a definition based on parameter-passing independence. Intuitively, the definition implies that functions are pure mappings from arguments to results; the operational decision of how to pass the arguments is irrelevant. In the context of Haskell, our definition is consistent with the fact that the traditional call-by-name denotational semantics coincides with the traditional call-by-need implementation. Furthermore, our definition is compatible with the stream-based, continuationbased, and monad-based integration of computational effects in Haskell. Finally, we observe that call-by-name reasoning principles are unsound in compilers for monadic Haskell.

Functional programming in non-functional languages

It is possible to employ a functional style of programming in languages that are not traditionally considered functional languages. Some non-functional languages have borrowed features such as higher-order functions, and list comprehensions from functional programming languages. This makes it easier to adopt a functional style when using these languages. Functional constructs such as higher-order functions and lazy lists can be obtained in C++ via libraries.^[15] In C one can use function pointers to get some of the effects of higher-order functions, for example one can implement the common function map using function pointers. Widespread declarative domain specific languages like SQL and Lex/Yacc, while not always Turing-complete, use some elements of functional programming, especially in eschewing mutable values.

Coding styles

Imperative programs tend to emphasize the series of steps taken by a program in carrying out an action, while functional programs tend to emphasize the composition and arrangement of functions, often without specifying explicit steps. A simple example of two solutions to the same programming goal (using the same multi-paradigm language Python) illustrates this.

# imperative style
target = [] # create empty list
for item in source_list: # iterate over each thing in source
    trans1 = G(item) # transform the item with the G() function
    trans2 = F(trans1) # second transform with the F() function
    target.append(trans2) # add transformed item to target

A functional version has a different feel to it:

# functional style
# FP-oriented languages often have standard compose()
compose2 = lambda F, G: lambda x: F(G(x))
target = map(compose2(F, G), source_list)

In contrast to the imperative style that describes the steps involved in building target, the functional style describes the mathematical relationship between source_list and target.

List of functional programming topics

Foundational concepts

• Programming paradigm
• Declarative programming
  • Declarative programming language
• Programs as mathematical objects
• Function-level programming
• Purely functional
• Lambda programming
• Static scoping
• Higher-order function
• Referential transparency

Lambda calculus

• Currying
• Lambda abstraction
• Church-Rosser theorem
• Extensionality
• Church numeral

Combinatory logic

• Fixed point combinator
• SKI combinator calculus
• B,C,K,W system
• SECD machine
• Graph reduction machine

Intuitionistic logic

• Sequent, sequent calculus
• Natural deduction
• Intuitionistic type theory
• BHK interpretation
• Curry-Howard
• Linear logic
• Game semantics

Type theory

• Typed lambda calculus
• Typed and untyped languages
• Type signature
• Type inference
• Datatype
• Algebraic data type
• Type variable
• First-class value
• Polymorphism
• Calculus of constructions

Denotational semantics

• Domain theory
  • Directed complete partial order
  • Knaster-Tarski theorem

Category theory

• Cartesian-closed category
• Yoneda lemma

Operational issues

• Graph reduction
  • Combinator graph reduction
• Non-strict programming language
• Lazy evaluation, eager evaluation
• Speculative evaluation
• Side-effect
• Assignment
  • Setq
• Continuation passing style
• Operational semantics
• State transition system
• Simulation preorder
• Bisimulation
• Monads in functional programming
• Exception handling
• Garbage collection (computer science)

Languages

• Clean programming language
• Erlang programming language
• FP programming language
• F# programming language,
• Haskell programming language
  • Glasgow Haskell Compiler
  • Hugs
  • Template Haskell
• ISWIM
• Kent Recursive Calculator
• Kogut programming language
• Lisp
  • AutoLISP
  • Common Lisp
  • Emacs Lisp
  • Scheme programming language
  • Lush programming language
• Mercury programming language
• Miranda programming language
• ML programming language
  • Category:ML programming language family
  • Objective Caml
  • Standard ML
• Scala
• SISAL

Curry: A Truly Integrated Functional Logic Language

Curry is a universal programming language aiming to amalgamate the most important declarative programming paradigms, namely functional programming and logic programming. Moreover, it also covers the most important operational principles developed in the area of integrated functional logic languages: "residuation" and "narrowing" (there is an older survey and a newer survey on functional logic programming).

Curry combines in a seamless way features from functional programming (nested expressions, higher-order functions, lazy evaluation), logic programming (logical variables, partial data structures, built-in search), and concurrent programming (concurrent evaluation of expressions with synchronization on logical variables). Moreover, Curry provides additional features in comparison to the pure languages (compared to functional programming: search, computing with partial information; compared to logic programming: more efficient evaluation due to the deterministic and demand-driven evaluation of functions).

The development of Curry is an international initiative intended to provide a common platform for the research, teaching and application of integrated functional logic languages. The design of Curry is mainly discussed in the Curry mailing list. A detailed report describing the language is available here. To get an idea of Curry, you may have a look into the short list of Curry's features or a tutorial on Curry. (more)

Microsoft moves on F# functional language

F# (pronounced F Sharp) is a multi-paradigm programming language, targeting the .NET Framework, that encompasses functional programming as well as imperative object-oriented programming disciplines. It is a variant of ML and is largely compatible with the OCaml implementation. F# was initially developed by Don Syme at Microsoft Research but is now being developed at Microsoft Developer Division and will be distributed as a fully supported language in the .NET Framework and Visual Studio ecosystem.

F# is a programming language that provides the much sought-after combination of type safety, performance and scripting, with all the advantages of running on a high-quality, well-supported modern runtime system. F# gives you a combination of (more)

Haskell

Haskell is an advanced purely functional programming language. The product of more than twenty years of cutting edge research, it allows rapid development of robust, concise, correct software. With strong support for integration with other languages, built-in concurrency and parallelism, debuggers, profilers, rich libraries and an active community, Haskell makes it easier to produce flexible, maintainable high-quality software. (more)

Simon Peyton Jones on Functional Programming and Haskell on www.se-radio.net

We start our discussion with a brief look at what Haskell is and how a pure functional language is different from non-pure languages. We then look at the basic building blocks and the philosophy of the language, discussing concepts such as the lambda calculus, closures, currying, immutability, lazy evaluation, memoization, and the role of data types in functional languages. A significant part of the discussion is then spent on the management of side effects in a pure language - in other words, the importance of monads. We conclude the episode with a look at Haskell's importance and community today.

[audio=http://media.libsyn.com/media/seradio/seradio-episode108-simonPeytonJonesOnFunctionalProgramming.mp3]

link

Objective Caml

Objective Caml (OCaml) is the main implementation of the Caml programming language, created by Xavier Leroy, Jérôme Vouillon, Damien Doligez, Didier Rémy and others in 1996. OCaml is an open source project managed and principally maintained by INRIA.

OCaml extends the core Caml language with object-oriented constructs.

OCaml's toolset includes an interactive toplevel interpreter, a bytecode compiler, and an optimizing native code compiler. It has a large standard library that makes it useful for many of the same applications as Python or Perl, as well as robust modular and object-oriented programming constructs that make it applicable for large-scale software engineering.

OCaml is the successor to Caml Light. The acronym CAML originally stood for Categorical Abstract Machine Language, although OCaml abandons this abstract machine. (more)

Coq

As it turns out Coq is really a dependently typed functional programming language masquerading as a proof assistant! I’ve spent quite a lot of time over the past few weeks writing total functional programs in Coq and proving properties about them. I’ve done a bunch of simple things so far, including a proof of correctness for various properties of insertion sort. I started with merge sort, but stalled when it got too complicated. I’m starting to get a feel for using Coq however and large proofs are getting much easier. (more)

Functional Programming with Python (RuPy 2008)

Python is a general-purpose, high-level programming language. Its design philosophy emphasizes programmer productivity and code readability. Python's core syntax and semantics are minimalist, while the standard library is large and comprehensive. It is unusual among popular programming languages in using whitespace as block delimiters.

Python supports multiple programming paradigms (primarily object oriented, imperative, and functional) and features a fully dynamic type system and automatic memory management; similar to Perl, Ruby, Scheme, and Tcl.

Python was first released by Guido van Rossum in 1991. The language has an open, community-based development model managed by the non-profit Python Software Foundation. While various parts of the language have formal specifications and standards, the language as a whole is not formally specified. The de facto standard for the language is the CPython.

[slideshare id=370124&doc=functionalpython-1208997222437657-8&w=425]

[www.wikipedia.org]
[http://caml.inria.fr/]
[http://www.informatik.uni-kiel.de/~curry,/]
[http://www.haskell.org/]
[http://www.infoworld.com]
[http://citeseerx.ist.psu.edu]
[http://www.rubrication.net/2007/03/07/coq/]
[http://research.microsoft.com/fsharp/fsharp.aspx]

Nous allons voir comment développer un client pour le logiciel serveur Memcached avec le langage Objective Caml. Le logiciel Memcached permet en résumé de stocker des données en mémoire vive ce qui offre des accès très rapides. Une valeur est simplement indexée par une clef unique qui est utilisée par le client pour écrire, modifier ou lire cette donnée. Memcached est typiquement utilisé pour stocker des valeurs temporaires afin de gérer un cache d'où son nom. Son protocole est relativement simple ce qui en fait un bon candidat pour une première approche du développement en Objective Caml.

Le code source complet est disponible dans cette archive.

• Protocole Memcached :

La documentation disponible sur le protocole donne un premier aperçu des fonctions qui seront nécessaires :

- Les commandes simples comme la demande de version du serveur ou le "flush_all" nécessitent simplement l'envoi d'une seule ligne de texte au serveur qui enverra en réponse une ligne de texte.

- Les commandes plus complexes comme les statistiques ou l'écriture d'une donnée nécessitent de gérer l'envoi ou la réception de plusieurs lignes.

- Il sera nécessaire de pouvoir vérifier si la réponse du serveur à une commande du client ne correspond pas à une erreur.

- Nous aurons évidemment besoin d'une fonction pour ouvrir et fermer la connexion au serveur Memcached.

- Chaque commande prévue par le protocole sera implémentée par une fonction dédiée.

• Typage :

Le protocole nous indique également les types à gérer. Il est par exemple souvent fait référence à des entiers non signés en 16, 32 ou 64 bits. Comme la notion d'entier non signé n'est pas disponible nativement en Objective Caml, nous allons gérer ces différents types par l'intermédiaire de modules dédiés.

Prenons l'exemple le plus simple qui est celui des entiers non signés 16 bits. La signature de ce module sera définie dans un fichier appelé "memcache_int16u.mli" :

[sourcecode language="java"]
type t
exception Overflow of t
val zero : t
val min : t
val max : t
val to_string : t -> string
val of_string : string -> t
val of_int : int -> t
[/sourcecode]

Le type "t" correspondra au type Objective Caml natif pour stocker l'entier. En l'occurrence, pour les entiers non signés 16 bits, il s'agira du type "int". L'exception "Overflow" sera levée si l'entier dépasse les limites. Ici ce sera le cas si le nombre est négatif ou bien dépasse la valeur 65535 (2^16 - 1).

On définie également les valeurs zéro (0), minimale (0) et maximale (65535). On aura également à notre disposition trois fonctions de conversions pour par exemple transformer une chaîne de caractères représentant un nombre vers le type entier non signé 16 bits.

Les trois modules dédiés à la gestion des entiers non signés auront cette même signature. Seule l'implémentation sera différente. Comme la signature de ce module reste très simple, il a été facile de la rédiger directement. Pour les signatures plus complexes, il est possible de générer automatiquement une première version à partir du fichier .ml d'implémentation grâce à l'option "-i" du compilateur Objective Caml.

Un autre type important correspond à la structure qui nous servira à stocker les informations pour la connexion au serveur :

[sourcecode language="java"]
type connection = {
host: string; (** adresse IP du serveur *)
port: int; (** numéro de port *)
in_channel: in_channel; (** canal de communication entrant *)
out_channel: out_channel; (** canal de communication sortant *)
};;
[/sourcecode]

Un enregistrement de ce type sera passé en paramètre à toutes les fonctions qui auront besoin d'échanger des données avec le serveur Memcached.

• Implémentation :

Passons maintenant au développement des fonctions proprement dites. Nous en détaillerons uniquement certaines pour donner un aperçu du langage Objective Caml.

Prenons par exemple la fonction chargée de recevoir plusieurs lignes en provenance du serveur :

[sourcecode language="java"]
(**
Reçoit plusieurs lignes du serveur.
@param connection Informations pour la connexion (déjà ouverte).
@return Une FIFO contenant les lignes dans l'ordre de réception + une chaîne de caractères concaténation des lignes reçues et séparées par \n.
*)
let receive_lines_from_server ~connection =
(* c'est par le canal de communicatio entrant que notre client reçoit les données du serveur *)
let in_channel = connection.in_channel in
(* on créé une structure FIFO pour stocker les lignes reçues du serveur *)
let queue_lines = Queue.create () in
(* on créé un tampon qui stockera lui aussi les lignes reçues du serveur mais sous forme de chaîne de caractères *)
let text_lines = Buffer.create 1024 in
(* on créé une variable "l" (une référence sur une chaîne vide) qui servira à stocker une ligne reçue *)
let l = ref "" in
(* on définie une fonction récursive "one_line" qui servira à traiter une ligne *)
let rec one_line () =
(* on récupère la ligne courante qui provient du serveur *)
l := input_line in_channel;
(* on supprime les fins de lignes *)
l := Str.global_replace regexp_trail_eol "" !l;
(* si ce n'est pas la dernière ligne *)
if (!l <> end_line) then (
(* on stocke la ligne courante dans la FIFO et dans le tampon *)
Queue.add !l queue_lines;
Buffer.add_string text_lines (!l ^ "\n");
(* on rappelle la fonction pour traiter la prochaine ligne *)
one_line ();
)
(* plus de ligne à traiter, on peut renvoyer le résultat *)
else (
(queue_lines, Buffer.contents text_lines)
)
in
(* il faut appeler une première fois la fonction "one_line" pour commencer le traitement *)
one_line ()
;;
[/sourcecode]

Notez que la variable "l" est une référence sur une chaîne vide dont la valeur est modifiée dans la fonction "one_line". Pour assigner une nouvelle valeur à une référence, il est nécessaire d'utiliser le signe ":=". Et pour obtenir le contenu (la valeur) de la référence, on utilise le signe "!" devant le nom de la variable.

La fonction renvoie un tuple (une liste de valeurs) dont le premier élément est la suite des lignes reçues du serveur. Il s'agit de la variable "queue_lines" qui est une file FIFO. Le deuxième élément renvoyé est une chaîne de caractères qui contient la concaténation des lignes reçues du serveur. Les deux éléments contiennent donc au final les mêmes données. L'intérêt de renvoyer ces données sous deux formes différentes est de pouvoir selon les besoins traiter la réponse du serveur soit comme une suite de lignes soit comme une chaîne de caractères.

Voyons ensuite la fonction qui envoie la commande pour stocker une valeur auprès du serveur :

[sourcecode language="java"]
(**
Commandes de stockage d'une clef/valeur.
@param cmd Commande (voir la section "Storage commands" du protocole).
@param key Clef de la valeur.
@param flag Valeur "flag" d'une commande de stockage (voir la section "Storage commands" du protocole).
@param time Expiration.
@param bytes Nb d'octets des données à stocker.
@param data Les données à stocker.
@param connection Informations pour la connexion (déjà ouverte).
@raise Bad_answer_from_server Réponse inattendue du serveur.
@return Indique si la valeur a été stockée ou pas.
*)
let storage ~cmd ~key ~flag ~time ~bytes ~data ~connection =
(* on construit la commande de stockage à partir des paramètres *)
let s = (cmd ^ " " ^ key ^ " " ^ (Int16u.to_string flag) ^ " " ^ (string_of_int time) ^ " " ^ (string_of_int bytes)) in
(* on envoie la ligne au serveur *)
let () = send_one_line_to_server ~connection ~s in
(* et ensuite on envoie au serveur la donnée à stocker *)
let s = Buffer.contents data in
let () = send_one_line_to_server ~connection:connection ~s:s in
(* le serveur doit nous retourner une réponse *)
let answer = receive_one_line_from_server ~connection:connection in
(* on vérifie que cette réponse ne correspond pas à une erreur *)
let () = check_error_answer_from_server ~answer:answer in
(* on examine ensuite cette réponse qui doit correspondre à une des réponses possibles pour la commande de stockage *)
let new_value =
match answer with
| "STORED" -> Stored
| "NOT_STORED" -> Not_stored
| _ -> raise (Bad_answer_from_server answer)
in
(* la fonction retourne cette réponse sous forme d'un type "storage_answer" que nous avons défini *)
new_value
;;
[/sourcecode]

La fonction prend en paramètres toutes les données nécessaires au traitement qu'elle effectue. Notez que le caractère "~" au début de chaque paramètre indique que ce sont des paramètres nommés. C'est à dire que lorsque la fonction sera appelée, il sera possible d'indiquer ce nom pour chaque valeur de paramètre ce qui permet ainsi de s'affranchir du respect de l'ordre de définition des paramètres. La fonction peut ainsi être appelée de cette manière :

[sourcecode language="java"]
storage ~connection:connexion ~flag:(Int16u.of_int 32) ~time:1219422874 ~bytes:1024 ~cmd:"commande" ~key:"clef" ~data:tampon
[/sourcecode]

On peut voir que l'on rappelle pour chaque paramètre son nom défini lors de la déclaration de la fonction. Il n'est donc pas obligatoire de respecter le même ordre dans les paramètres. Si le nom des paramètres est suffisamment explicite, c'est également une manière de rendre plus clair le code.

Notez que le caractère "^" est utilisé pour la concaténation des chaînes de caractères. C'est l'équivalent du caractère "." dans le langage PHP.

Pour finir, remarquez une construction très utilisée dans le langage Objective Caml qui est le "match". Ici, cela permet de comparer la valeur de la variable "answer" avec les chaînes de caractères "STORED" et "NOT_STORED" et selon le cas de stocker dans "new_value" soit "Stored" soit "Not_stored". Si la comparaison échoue, le défaut indiqué par le caractère "_" est de lever l'exception "Bad_answer_from_server". La construction "match" ne s'applique pas uniquement à des chaînes de caractères. Elle peut par exemple s'appliquer à des listes ou à un type défini dans le programme. Vous êtes invité à consulter la documentation du langage Objective Caml pour plus de détails.

Pour finir, voyons la fonction qui permet d'obtenir les statistiques globales du serveur :

[sourcecode language="java"]
(**
Statistiques globales du serveur.
@param connection Informations pour la connexion (déjà ouverte).
@raise Bad_answer_from_server Si réponse inattendue ou si une des réponses n'est pas dans la liste "names_stats".
@return Une liste de paires (nom, valeur) correspondant aux statistiques.
*)
let stats ~connection =
(* on envoie la commande "stats" au serveur *)
let s = "stats" in
let () = send_one_line_to_server ~connection:connection ~s:s in
(* on reçoit la réponse en plusieurs lignes de la part du serveur *)
let (lines, answer) = receive_lines_from_server ~connection:connection in
(* on doit au moins recevoir une ligne *)
(* si ce n'est pas le cas, on lève une exception *)
let () =
if (Queue.length lines <= 0) then (raise (Bad_answer_from_server answer))
in
(* fonction qui traite une seule ligne des statistiques reçues *)
let one_line acc l =
(* les lignes sont de la forme "STAT " *)
let r = Str.regexp ("^STAT \\([^ ]+\\) \\(.+\\)") in
(* si la ligne est au bon format *)
if (Str.string_match r l 0) then (
try
(* on récupère la partie "nom" *)
let name = Str.matched_group 1 l in
(* on récupère la partie "valeur" *)
let value = Str.matched_group 2 l in
(* on vérifie que les clefs/valeurs reçues sont bien autorisées *)
let () =
if (not (List.mem_assoc name names_stats)) then (raise (Bad_answer_from_server answer))
in
(* conversion des valeurs en entiers non signés *)
let new_value =
let type_value = List.assoc name names_stats in
match type_value with
| Ts_32u _ -> Ts_32u (Int32u.of_string value)
| Ts_64u _ -> Ts_64u (Int64u.of_string value)
| Ts_string _ -> Ts_string value
| Ts_2x_32u _ -> Ts_2x_32u value
in
(* on ajoute le tuple (nom, valeur) à la liste qui sera renvoyée par la fonction *)
(name, new_value) :: acc
with
| Not_found -> raise (Bad_answer_from_server answer) (* une des lignes n'est pas au bon format *)
)
else (
raise (Bad_answer_from_server answer) (* une des lignes n'est pas au bon format *)
)
in
(* pour chaque ligne reçue du serveur, on appelle la fonction "one_line" *)
Queue.fold (one_line) [] lines
;;
[/sourcecode]

On peut voir que la fonction "one_line" a notamment comme paramètre une variable appelée "acc" (pour "accumulateur"). Ce type de paramètre est souvent présent en Objective Caml. Il permet en effet d'accumuler le résultat d'appels successifs à une fonction donnée. Ici, la fonction "one_line" est appelée autant de fois qu'il y a de lignes à traiter grâce à l'utilisation de la fonction "Queue.fold". On peut également voir dans l'appel à cette fonction que l'accumulateur a comme valeur de départ une liste vide.

On retrouve aussi la construction "match...with" dont nous avons déjà parlé. Cette fois-ci, la comparaison porte sur les types d'entiers non signés que nous avions définis.

• Compilation "bytecode" :

Objective Caml offre deux modes de compilation :

- Un mode qui permet d'obtenir un exécutable natif, de la même manière que l'on obtiendrait un exécutable depuis des sources en langage C. L'avantage est de produire un programme avec de très bonnes performances. En contre-parti, ce programme fonctionnera uniquement sur la plate-forme (par exemple un système FreeBSD sur PC) pour lequel il a été compilé.

- Un mode "bytecode" qui produira un programme destiné à être exécuté par la machine virtuelle Objective Caml. C'est un mécanisme similaire à celui utilisé par le langage Java. L'exécution sera plus lente qu'avec un programme natif mais il ne sera plus dépendant de la plate-forme.

Vous trouverez dans l'archive contenant les sources un script shell indiquant les commandes pour compiler le code en mode "bytecode" et ainsi obtenir une librairie et un exécutable de démonstration.

Par exemple, pour compiler le fichier memcache_int16u.ml qui contient l'implémentation des entiers 16 bits non signés, la commande est la suivante :

[sourcecode language="java"]
ocamlc -c memcache_int16u.ml
[/sourcecode]

Ce qui produira un fichier objet "memcache_int16u.cmo".

• Références :

Protocole Memcached : http://code.sixapart.com/svn/memcached/trunk/server/doc/protocol.txt

Site officiel Objective Caml : http://www.ocaml.org/

- Try to build the queries fighting with raw strings.
- Or use CAML .Net.

CAML .Net is a personal project, from CodePlex, that gives to you the possibility to build the queries, in a very easy way, using object oriented programming. Just fantastic.

To take a look on CAML .Net, you can click here.

Так же тулза из раздела "мастхев" для wss девелоперов. Позволяет просмотреть структуру списков, представлений, а так же выполнять и генерировать код запросов для приложений.

У ребят имеются очень полезные наработочки по шарику. Одна из них используется мною практически каждый день. Называется приспособление U2U Caml Query Builder 2007 v.3.1. (Качать тут http://www.u2u.info/SharePoint/U2U%20Community%20Tools/Forms/AllItems.aspx). Тулза позволяет генерить и отлаживать CAML запросы.

So after having my few hours I created this handy reference sheet useful at many places.

You can use this sheet at many places like "BaseType" values (just below) are always useful as part of SPSiteDataQuery :

<Lists BaseType='0' MaxListLimit='0'/> or similar goes for "Server Template"

<Lists ServerTemplate='303' MaxListLimit='0'/>

Another handy example is "FieldType" again which is useful in CAML queries.

<FieldRef Name='MyLookupField'/><Value Type='Lookup'>

Endless possibilities. Be patient to load this huge page (And sincere apology for SMALL fonts, hard to fit in screen)

Don't forget to comment back If I am missing something that should be here too .