Nacho Martin

sf2vigo 2011

2011-12-18T00:00:00+01:00

Los amigos de Galpon tuvieron la amabilidad de invitarme a las Jornadas Symfony2 2011 de Vigo. Amabilidad es la palabra justa porque fueron amabilísimos. ¡Muchas gracias, Susana, Paz, Miguel y resto de galponeros!

Cuelgo aquí los vídeos de mis charlas:

Doctrine 2

Doctrine2 sf2Vigo

View more presentations from Nacho Martín

Microframework Silex

Git & Github

Presentacion git

View more presentations from Nacho Martín

Desymfony 2011

2011-07-05T00:00:00+02:00

Si el año pasado tuve la suerte de dar una charla sobre Doctrine en las Jornadas Symfony 2010, este ha sido aún mejor. Formé parte del equipo organizador de desymfony junto a David Castelló, Javier Eguiluz, Albert Jessurum, Marcos Labad y Javier López. Di dos charlas (el Modelo, Silex) y un taller de Git junto a Albert. Mucho trabajo, pero ¡muy agradecido!

El ambiente durante el evento fue extraordinario durante los tres días, que fueron intensísimos porque duraban desde la mañana hasta la hora que pudieses aguantar en un pub. Muchas gracias a los asistentes por hacer que el evento fuera una gozada y ¡nos vemos en 2012!

Aquí dejo los vídeos de mis presentaciones:

El modelo con Doctrine2

desymfony 2011 - El Modelo. Doctrine2 from decharlas.com on Vimeo.

El código fuente de esta aplicación de ejemplo que desarrollamos está publicado en github.

Silex

desymfony 2011 - Microframework Silex from decharlas.com on Vimeo.

El código fuente de la aplicación Silex que uso para estas transparencias está publicado en github.

Writing a new Doctrine2 extension

2011-05-06T19:02:11+02:00

In this post I will explain how to create a new Doctrine2 extension. Extensions fulfill the role played in Doctrine1 by behaviors.

In a app that I am currently developing I faced the following situation:

People can create projects. A project can have tasks, comments, etc.
Projects have a version number.
When the client is browsing the project, he makes calls every few seconds saying: I am in version X.
When the client does an action (comment, open a new task...), it sends "I have just performed this action: blablabla"
When the server receives a new action from the client, it increments the version of the project.
When the server receives a "I am in version X" message and the version in the server is greater than X, it answers back saying: "user Y did action blablabla". And the client will hopefully make use of this information to update his copy of the project using backbone.js or whatever.
When the server receives a "I am in version X" and the version in the server is X, it answers: "Ok, you are fine".

So, in this scenario, what I need is an entity (History) that contains the version number of the project (that I will call status instead of version to avoid name collisions and confusions with other extensions), the id of the project, and a field to store the action messages. And I would like Doctrine to take care of the version number and increment it whenever I insert a new History entity. It is a simple behavior extension.

Why not simply using the versionable extension? Because versionable stores new versions of the entities, and I just want to store a description of the changes in between versions.

In order to start, I recommend grabbing the code https://github.com/l3pp4rd/DoctrineExtensions for two reasons: it has everything you need to start and test your code and it contains lots of examples on writing extensions.

Tests

Let's start writing some tests, to have a clear idea of the requirements that what we need to fulfill.

<?php
//tests/Gedmo/Historyable/HistoryableEntityTest.php
namespace Gedmo\Historyable;

use Tool\BaseTestCaseORM;
use Doctrine\Common\EventManager;
use Doctrine\Common\Util\Debug,
    Historyable\Fixture\Entity\History;

class HistoryableEntityTest extends BaseTestCaseORM
{
    const HISTORY = 'Historyable\Fixture\Entity\History';
    protected function getUsedEntityFixtures()
    {
      return array(
        self::HISTORY
      );
    }

    protected function setUp()
    {
        parent::setUp();

        $evm = new EventManager;
        $this->HistoryableListener = new HistoryableListener();
        $evm->addEventSubscriber($this->HistoryableListener);
        $this->em = $this->getMockSqliteEntityManager($evm);
    }

    public function testHistoryable()
    {
        $statusRepo = $this->em->getRepository(self::HISTORY);
        $this->assertEquals(0, count($statusRepo->findAll()));

        $status0 = new History();
        $status0->setResourceId(1);
        $status0->setAction("added stuff");
        $this->em->persist($status0);
        $this->em->flush();

        $statusLast = $statusRepo->getLast(1);
        $this->assertNotEquals(null, $statusLast);
        $this->assertEquals(0, $statusLast->getStatus(), "first status is 0");

        $status1 = new History();
        $status1->setResourceId(1);
        $status1->setAction("added more stuff");
        $this->em->persist($status1);
        $this->em->flush();

        $statusLast = $statusRepo->getLast(1);
        $this->assertEquals(1, $statusLast->getStatus(), "after saving two status of the same resource, last status is 1.");

        $status2 = new History();
        $status2->setResourceId(2);
        $status2->setAction("stuff done to a different resource");
        $this->em->persist($status2);
        $this->em->flush();

        $statusLast = $statusRepo->getLast(1);
        $this->assertEquals(1, $statusLast->getStatus(), "after saving 1 status to another resource, last status is 1");
    }
}

Is it clear enough? First two functions help us to set up the environment, to be able to flush entities to the database, and have our HistoryableListener listening, and the last one contains our tests. We test three tings: that the first status will be 0, that the status is incremented when we create actions for the same resource, and that actions created to another resource don't affect the last status count of this resource.

We need a fixture:

<?php
//tests/Gedmo/Historyable/Fixture/Entity/History.php
namespace Historyable\Fixture\Entity;

/**
 * @Entity(repositoryClass="Historyable\Fixture\Repository\HistoryRepository")
 * @gedmo:Historyable
 */

class History
{
    /**
     * @Id
     * @GeneratedValue
     * @Column(type="integer")
     */
    private $id;

    /**
    * @gedmo:refVersion
    * @Column(name="resource_id", type="integer")
    */
    private $resourceId;

    /**
     * @gedmo:status
     * @Column(type="integer")
     */
    private $status;

    /*
     * @Column(type="string")
     */
    private $action;

    /*
    * ... setters and getters ...
    */
}

We are using three custom annotations: @gedmo:Historyable marks this entity as historyable, @gedmo:status tells our extension that that field contains the status (or versions) and @gedmo:refVersion tells our extension that that field contains the id of the entity to which this status refers.

And the repository:

<?php
//tests/Gedmo/Historyable/Fixture/Repository
namespace Historyable\Fixture\Repository;

use Gedmo\Historyable\Entity\Repository\BaseHistoryRepository;

class HistoryRepository extends BaseHistoryRepository
{

}

Just an empty repository extending BaseHistoryRepository, that we will define later.

With this code, we have our tests prepared.

Mapping

We need some code to take care of these new annotations. Three definitions:

<?php
//lib/Gedmo/Historyable/Mapping/Annotations.php
namespace Gedmo\Historyable\Mapping;
use Doctrine\Common\Annotations\Annotation;

final class Historyable extends Annotation
{
}

final class RefVersion extends Annotation
{
}

final class Status extends Annotation
{
}

Now, we need a driver to transform these annotations to configuration options. This class checks that the annotations are not wrong. For instance, it checks that there is only one status field, one refVersion field, that they are integers, and so on. I won't paste the full code here, (you can check the full source code here), because with all the checks it is quite long and repetitive.

<?php
//lib/Gedmo/Historyable/Mapping/Driver
namespace Gedmo\Historyable\Mapping\Driver;

use Gedmo\Mapping\Driver,
    Doctrine\Common\Annotations\AnnotationReader,
    Gedmo\Exception\InvalidMappingException;

class Annotation implements Driver
{
    /**
     * Annotation to define that this object is historyable
     */
    const HISTORYABLE = 'Gedmo\\Historyable\\Mapping\\Historyable';
    /**
     * Annotation to define that this property is a status
     */
    const STATUS = 'Gedmo\\Historyable\\Mapping\\Status';
    /**
     * Annotation to define that this property is a refVersion
     */
    const REFVERSION = 'Gedmo\\Historyable\\Mapping\\RefVersion';
    public function readExtendedMetadata($meta, array &amp;$config)
    {
        require_once __DIR__ . '/../Annotations.php';
        $reader = new AnnotationReader();
        $reader->setAnnotationNamespaceAlias('Gedmo\\Historyable\\Mapping\\', 'gedmo');

        $class = $meta->getReflectionClass();
        // class annotations
        $classAnnotations = $reader->getClassAnnotations($class);
        if (isset($classAnnotations[self::HISTORYABLE])) {
            $config['historyable'] = true;
        }
        // property annotations
        foreach ($class->getProperties() as $property) {
            if ($meta->isMappedSuperclass &amp;&amp; !$property->isPrivate() ||
                $meta->isInheritedField($property->name) ||
                isset($meta->associationMappings[$property->name]['inherited'])
            ) {
                continue;
            }
            if ($status = $reader->getPropertyAnnotation($property, self::STATUS)) {
                $field = $property->getName();
                if (!$meta->hasField($field)) {
                    throw new InvalidMappingException("Unable to find status [{$field}] as mapped property in entity - {$meta->name}");
                }
                $config['status_field'] = $field;
            }
            if ($refVersion = $reader->getPropertyAnnotation($property, self::REFVERSION)) {
                $field = $property->getName();
                $config['refVersion_field'] = $field;
            }
        }
    }
}

Now we define an Adapter that will contain the logic needed to get the new status number.

First, we write an interface for the adapter. This interface specifies the functions that the adapter must implement. This is because you will have usually two adapters, one for ORM (relational databases) and one for ODM (MongoDb). In this case, I will implement the ORM, but having an interface for the adapter is a good practice. It contains only a function:

<?php
//lib/Gedmo/Historyable/Mapping/Event
namespace Gedmo\Historyable\Mapping\Event;
use Gedmo\Mapping\Event\AdapterInterface;

interface HistoryableAdapter extends AdapterInterface
{
    /**
     * Get new status number
     *
     * @param ClassMetadata $meta
     * @param object $object
     * @return integer
     */
    function getNewStatus($config, $meta, $object);
}

This function will be the only and killer feature of our simple extension :). How does this function look like?

<?php
namespace Gedmo\Historyable\Mapping\Event\Adapter;
use Gedmo\Mapping\Event\Adapter\ORM as BaseAdapterORM;
use Gedmo\Historyable\Mapping\Event\HistoryableAdapter;

final class ORM extends BaseAdapterORM implements HistoryableAdapter
{
    public function getNewStatus($config, $meta, $object)
    {
        $em = $this->getObjectManager();
        $objectMeta = $em->getClassMetadata(get_class($object));
        $identifierField = $this->getSingleIdentifierFieldName($objectMeta);
        $objectId = $objectMeta->getReflectionProperty($identifierField)->getValue($object);

        //Real stuff
        $dql = "SELECT MAX(status.{$config['status_field']}) FROM {$meta->name} status";
        $dql .= " WHERE status.{$config['refVersion_field']} = :objectId";

        $q = $em->createQuery($dql);
        $q->setParameters(array(
            'objectId' => $object->getResourceId()
        ));
        $currver = $q->getSingleScalarResult();
        return $currver != null ? $currver + 1: 0;
    }
}

All this work for a single query that will tell us what will be the next status number of this History! Check out how we can make use of the array $config, that holds the names of the fields that are relevant to us, to build the query.

Event Listener

And now what? We are close to the end. We have a function that will tell us what the next status number must be, but how can we set this field whenever a new Historyable entity is inserted? Now is when the Event Subscriber comes to help us.

<?php
//lib/Gedmo/Historyable/HistoryableListener.php
namespace Gedmo\Historyable;

use Gedmo\Mapping\MappedEventSubscriber,
    Gedmo\Historyable\Mapping\Event\HistoryableAdapter,
    Doctrine\Common\EventArgs;

class HistoryableListener extends MappedEventSubscriber
{
    //we want to subscribe to these two events
    public function getSubscribedEvents()
    {
        return array(
            'onFlush',
            'loadClassMetadata',
        );
    }

    /**
     * Mapps additional metadata
     *
     * @param EventArgs $eventArgs
     * @return void
     */
    public function loadClassMetadata(EventArgs $eventArgs)
    {
        $ea = $this->getEventAdapter($eventArgs);
        $this->loadMetadataForObjectClass($ea->getObjectManager(), $eventArgs->getClassMetadata());
    }


    /**
     * Looks for historyable objects being inserted 
     *
     * @param EventArgs $args
     * @return void
     */
    public function onFlush(EventArgs $eventArgs)
    {
        $ea = $this->getEventAdapter($eventArgs);
        $om = $ea->getObjectManager();
        $uow = $om->getUnitOfWork();

        //For each object being inserted...
        foreach ($ea->getScheduledObjectInsertions($uow) as $object) {
            $meta = $om->getClassMetadata(get_class($object));
            $config = $this->getConfiguration($om, $meta->name);
            //we get the new status number and set it!
            $status = $ea->getNewStatus($config, $meta, $object);
            $object->setStatus($status);
            $ea->recomputeSingleObjectChangeSet($uow, $meta, $object);
        }
    }
}

We will listen to the onFlush event, retrieve the objects that are marked for insertion (because they are new), and compute the status number based on the refVersion field. With this, we don't have to care about updating manually the status field anymore.

Repository

Now, won't it be useful to have a function in our repository that retrieves the last status for a project? We can make a repository class and make our real repositories (like the one that we wrote in fixtures) to extend this, having access to that function:

<?php
//lib/Gedmo/Historyable/Entity/Repository/BaseHistoryRepository.php
namespace Gedmo\Historyable\Entity\Repository;

use Doctrine\ORM\Query,
    Doctrine\ORM\EntityRepository,
    Doctrine\ORM\EntityManager,
    Doctrine\ORM\Mapping\ClassMetadata;

class BaseHistoryRepository extends EntityRepository
{
    /**
     * Historyable listener on event manager
     *
     * @var AbstractTreeListener
     */
    protected $listener = null;

    //This checks that we are in a Historyable Entity
    public function __construct(EntityManager $em, ClassMetadata $class)
    {
        parent::__construct($em, $class);
        $histListener = null;
        foreach ($em->getEventManager()->getListeners() as $event => $listeners) {
            foreach ($listeners as $hash => $listener) {
                if ($listener instanceof \Gedmo\Historyable\HistoryableListener) {
                    $histListener = $listener;
                    break;
                }
            }
            if ($histListener) {
                break;
            }
        }
        if (is_null($histListener)) {
            throw new \Gedmo\Exception\InvalidMappingException('This repository can be attached only to ORM historyable listener');
        }
        $this->listener = $histListener;
    }
    
    //Our beloved getLast function
    public function getLast($id){
        $meta = $this->getClassMetadata();
        $config = $this->listener->getConfiguration($this->_em, $meta->name);
        $qb = $this->_em->createQueryBuilder();
        $qb->select('st')
            ->from($meta->name, 'st')
            ->orderBy('st.'.$config['status_field'], 'DESC')
            ->setMaxResults(1);
        $q = $qb->getQuery();
        return $q->getSingleResult();
    }
}

And that is all! We have just finished our extension! Now we can run the tests and see if everything went fine.

I hope this helps you somehow!

Playing with Node.js

2011-03-17T00:47:43+01:00

I have been playing around with node.js lately. The pet project I did to learn it (available on github) is quite simple, but useful to me: I have a growing collection of hundreds of images in my hard drive and I need a system to bookmark new ones without mantaining a nightmare of folders and files with strange names.

This apps allows me, when I am browsing a website, to right-click on any image, and save it to the app using a Firefox extension. It saves them with a title and some tags, and then on the website of the app, I can see the collection of images, optionally filtered by tags. The resulting look is quite similar to Tumblr. At the moment it doesn't have support for different users/collections. As simple as that.

I used redis as database, express as framework, and jade as Well, enough preamble. Let's see some code. I am not going to comment every single line. You can access the repository for that. Only some parts that I think can give you a feeling on how is node.js, express and redis. Also, I want to write my impressions on different topics I have faced with this project, so I think a bit more about them and hopefully help somewhat other coders.

Retrieving a file and downloading it

  //retrieve it
  var theurl = http.createClient(80, host);
  var requestUrl = urlfile;
  console.log(&quot;Downloading file: &quot; + filename);
  var request = theurl.request('GET', requestUrl, {&quot;host&quot;: host});
  request.end();

  var dlprogress = 0;

  request.addListener('response', function (response) {
    var downloadfile = fs.createWriteStream(&quot;public/images/&quot;+filename, {'flags': 'a'});
    console.log(&quot;File size &quot; + filename + &quot;: &quot; + response.headers['content-length'] + &quot; bytes.&quot;);
    response.addListener('data', function (chunk) {
      dlprogress += chunk.length;
      downloadfile.write(chunk, encoding='binary');
    });
    //download finished
    response.addListener(&quot;end&quot;, function() {
     ///... store images, make thumbnails... do more stuff
      console.log(&quot;Finished downloading &quot; + filename);
    });
  });
});

This was quite cool. Being mainly a PHP developer I like how this works. It is event driven, so you can have a callback called while the downloading is in progress and write some code to be executed when the download is finished. I really like this, and I will try to keep it in mind if I have to write, for example, an app that has to process videos or some other big stuff without making the user wait until you do all your stuff, which is terrible. You can do things really quickly using this.

The nesting "problem"

Being node.js event-driven, you get used very quickly to nest and sometimes over-nest code. For instance, when you want to retrieve a image object from the database, you do something like:

  client.get( 'image:'+id, function( err, data ) {
    //check if there is no data
    //[...]
    var obj = JSON.parse( data.toString() );
    res.render('single.jade', {
      locals: {
        image: obj,
        tags: allTags
      }
    });

So, the function 'get' accepts as an argument the callback that will be called with the result of that 'get'. But, meanwhile, the code below the 'get' will be executed. Therefore, instead of writing sequentially what you whant to do with the data, you have to place that code in the callback. This pattern is used constantly, and you have soon callbacks nested into another callbacks, so you need to rewrite the code to make it more manageable. I am still a novice at this art ;)

But sometimes, nesting callbacks is not enough. For instance: what happens if you want to retrieve a list of images in a for loop and send the result to the view? You can not use the callback of every get, because in that case you don't have the full list. You must wait to show the list until the last object is retrieved. It took me a while to find out how to overcome this.

        var count = data.length;
        for (var i = 0; i &lt; data.length; i++) {
          var id = data[i].toString();
          client.get( 'image:'+id, function( err, dataIm ) {
            var obj = JSON.parse( dataIm.toString() );
            imagArr.push( obj);
            count--;
            if (count == 0){
              res.render('index.jade', {
                locals: {
                  images: imagArr.reverse(),
                  tags: allTags,
                  pagination: pagination
                }
              });
            }
          });
        }

Here we keep a 'count' variable that makes us possible to know if we are in the last iteration of the loop. Then, if that is the case, we render the view with the result. It is how the language is designed. It is quite possible that there is a more elegant way to solve this, but I haven't found it and this works. I am not sure if I feel comfortable about it. Please, use the comments to suggest a better solution.

About Redis

I had never used Redis before, and I am very glad I have tried it. It is not a Swiss knife, but it is very good doing what it has been designed for: a key-value data store. I have used Mongo before to store recipes in trecetas.com, and it is not so different as using SQL: you store documents and you can write complex queries against your database to retrieve them. Redis, on the other hand, makes you think in a very different way. In this app I used redis to have key-values for the images, being the key 'image:id' and the value a json object with the filename, the title and the description. Also, I have a list of all the images' ids in the key 'images', a similar list for each tag 'tag:id' and a 'tags' list with all the tags.

So Redis is basically for me a pool where I can store keys and values. But most importantly, if you decide to use Redis in a project, you are forced to think differently as you would do in SQL or Mongo. You are not expected to just store the tags in the json value of a key 'image:id', because you cannot query them. Instead, you build a new pool for each tag. If wanted to have multi-user support, I would have to maintain a new pool with the images for that user.

So, as I said, it is not a Swiss knife, but it is good to have a tool like this in case you need a lighting fast key-value storage.

Express

I used the express framework for this project. If you are, like me, used to complex frameworks like Symfony, Rails or Django, and you want to try node.js, you will find that express is not the same kind of thing (node.js is a much younger platform!). It is way much simpler and you will have to extend it to make something more complex than this pet project of mine. With that said, the things the framework does, it does them well. You have a simple but nice routing system based on regular expressions, logging, support for several templating engines, and you can use things like 'less' with no trouble.

Being as lightweight as it is, you are faced with a range of choices for your specific needs in case you want for instance a way to keep your server running after a crash, ORMs, templating engines, and so on. It is quite refreshing if you come, like me, from Symfony, where you have almost everything you need chosen for you, to play with different small modules and to combine them.

To sum up

I am by no means a guru, but if you want the opinion of some guy that is not looking for the perfect beauty in programming nor for the perfect tool, here it goes: When I was programming in node.js I felt refreshed. It is very nice to work with a young environment and the community seems to be working on a lot of nifty tools that give you some ideas even if you go back to your favourite framework next week. It took me a while to figure out how to deal with asynchronous programming, and I feel it can be difficult to solve some problems. That means for me that it would be difficult to convince my work colleges to work with node, but on the other side it has some nice advantages (speed is clearly one of them) that makes it very worthwhile to play with.

It also motivated me to read the book and watch the videos about Javascript by Douglas Crockford, that are really (really!) worthwhile.

Hint to avoid memory exhausted error (symfony + doctrine1.2)

2010-09-01T14:16:45+02:00

It is usual to have tasks that load a lot of objects from the database and do something with them. It can be generating a sitemap, generating thumbnails, etc.

When you do that, it is very likely that you find memory exhaustion problems like:

Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 71 bytes) in /blablabla/lib/vendor/symfony/lib/log/sfVarLogger.class.php on line 170

How to overcome this? Luckily I found this thread on the symfony group. If I had discovered that thread earlier it would have saved a lot of time, so I'm replicating the solution here :)

The problem is that the doctrine profiler is enabled for sfBaseTask, as sf_debug is set to true.

To disable the profiler, you have to create a new environment in config/databases.yml:

task:
  symfony:
      param:
        profiler: false

all:
  symfony:
    class: sfDoctrineDatabase
      param:
        dsn:   ....
        username:   ...
        password:   ...

And then run the task with the option --env=task

De vuelta de las Jornadas Symfony

2010-07-08T19:27:00+02:00

Esta semana he estado en las jornadas de symfony en la Universitat Jaume I de Castellón, que tuvieron la amabilidad de invitarme como ponente.Tenéis una crónica muy completa de Javier Eguiluz en symfony.es. Fue un éxito de asistencia, participación y organización. David Castelló merece mención especial por lo mucho que se lo curró. Dejo aquí las transparencias de mi presentación sobre Doctrine y más abajo el vídeo.

Doctrine - Nacho Martín - Jornadas Symfony en Castellón. Julio 2010

ORM Doctrine - decharlas 2010 from decharlas.com on Vimeo.

Automating the creation of new symfony 1.4 projects (doctrine + git)

2010-05-15T23:15:41+02:00

Sometimes I want to create new symfony projects to try something, or, well, sometimes even to do some actual work.

Then, I go to the Jobeet tutorial and follow the first steps. After that, since I use git, I check out this post to setup symfony as a submodule.

I have written a simple bash script with that post and a few more lines, to automatize the process. It may be useful for others, and for me to have it accesible, so I will post it here:

setupsymfony.sh:

#!/bin/bash

SYMFREPO=";git://symfony-git.git.sourceforge.net/gitroot/symfony-git/symfony-git";
DBUSER=";root";
DBPASS=";";
VENDOR_DIR=";lib/vendor/symfony";

if [ -z ";$DBPASS"; ]; then
 echo edit setupsymfony.sh to add your mysql password on line 4: DBPASS=\";password\";
 exit
fi

if [ -z ";$1"; ]; then
 echo usage: $0 project name
 exit
fi

mkdir $1
cd $1

git init
git clone $SYMFREPO $VENDOR_DIR
git submodule add $SYMFREPO $VENDOR_DIR/

git submodule foreach 'git submodule init &amp;&amp; git submodule update'

git commit -m 'Initial commit; initialized symfony-git (1.4)'

$VENDOR_DIR/data/bin/symfony generate:project $1
echo ";config/databases.yml"; &> .gitignore
echo ";cache/*"; &>&> .gitignore
echo ";log/*"; &>&> .gitignore

./symfony project:permissions
git add .
git commit -m 'Initialized symfony project'

./symfony generate:app frontend
git add .
git commit -m 'Initialized frontend application'

mysqladmin -u$DBUSER -p$DBPASS create $1
./symfony configure:database ";mysql:dbname=$1;hostname=localhost"; $DBUSER $DBPASS

usage: ./setupsymfony.sh name_of_the_new_project

remember to write your database password in the line 4 of the script.

Edit: updated to include the changes proposed by kbsali. Thanks!

Edit2: I have posted it in a gist.

List & Check, an open source symfony project

2010-05-15T14:23:27+02:00

I have just released a pet project I have been developing lately. It is called List & check, and it is about making (beautiful) lists easily.

What it is about

I use lists for almost everything: things I want t o try, things I want to do this week, books to read, movies to watch, places to visit, steps to do before a deployment... and since I saw the beautiful symfony-check website, I just wanted to be able to create lists with that beautiful look and great user experience.

I think that other list fans may enjoy it.

Things I wanted to try

I wanted to use this pet project as playground to try some things:

I used the Doctrine ORM, because I was using before Propel. I think that the change is worthwile only looking at the amount of documentation available for each ORM. I used the csDoctrineActAsSortablePlugin to make items sortable and it worked smoothly.
Redmine as a replacement of trac and pivotal tracker. I hate the installation of trac, Redmine gives me just what I wanted with a good user interface. It integrates well with git as well.
Git for Symfony. I am big fan of git, but I have had problems in the past working with Symfony and the huge amount of svn:externals you have to deal with. I have tried several scripts for doing the job, but none of them worked properly, and I had to tweak them... But, thanks to vjousse, that has placed a Symfony git repository in github, this time was very easy.
Ajaxized edit view. I wanted to have only a view both for editing the lists and for showing them better than having separate views for these actions. I think that for this project it makes sense to have everything in one place and the user experience is better, as in projects like Pivotal Tracker.
Focus in user experience. Related to the previous point, it is important that you can create lists, create new items, sorting and deleting them quickly, just writing the title and pressing enter over and over. At the same time, I wanted to make possible to detail the items. It may be stupid for a list of places to visit or a to-do list for this evening, but it makes sense for lists like steps to set up a workstation. This is a feature I miss in Tadalist. So now I can create lists quicky and as detailed as I want, given the markdown editor. I used also jquery-ui that made my life easier.
Markdown editor. Markdown is just great for creating content, and was very useful as format for the items' description.
Selenium. In my roadmap I want to create selenium tests for the whole site. I have not done it yet, because I have been tuning the user interface until the last moment, but looks like the perefect project to test with Selenium.

It is Open source

The full code is hosted at github. The idea is more to give ideas to other people for doing their websites, but if you want you can set up a list&check server. I have learnt a lot from the source code of symfonians and other OS projects, and I wanted to give something back to the community.

My experience with Diem CMS

2010-05-14T20:57:57+02:00

Lately, I wanted to do a website for a non profit association in which I am involved. Since I really like symfony, I decided to do it using one of the CMS that extend symfony. Looks like there are three main choices: Apostrophe, Diem and Sympal.

I tried first Apostrophe, because I liked a lot the demo video they have in their page. The concept is awesome and it is really worth a shot. Unfortunatelly, we wanted to have a blog, and the blog plugin for apostrophe is not ready yet. I had to abandon it, although I want to use it back in the future.

Then, I asked other symfonians on Twitter (@davidcastello, @loalf & @ivandebenito are always very helpful :), and, after seeing the demo on the Diem site, I installed it and followed the tutorial A Week of Diem Ipsum.

The tutorial is just great. Well explained and illustrative, and the CMS itself suited my needs, although I was disappointed when I saw that the last lesson ends with "On the next chapter we will add comments to the blog.", and... there is not next lesson. Anyways, I asked on IRC and they pointed me to dmCommentPlugin, which is the way to do it. #EDIT: That chapter of the tutorial is now available! :)

After following the tutorial you can notice that Diem gives what they promise:

You don't need to develop all those boring thinks common to lots of pages. They are already implemented, as you can expect from a CMS, but moreover everything works smoothly.
The wysiwyg editor of layout of pages is great. It makes the task of designing the layout almost pleasing (I hate dealing with CSS).
Extending it is quite easy, if you know symfony.

I created two new widgets for the landing page, a sort of squares with rounding boxes, a background image and some text inside... not very complicated, but it was easy, although I had to dig a bit in the code of already implemented widgets to solve some issues. I suppose that the documentation will grow and this won't be necessary in the future, but anyways it is not a big pain. It was quite easy. After all, I was new in Diem.

I developed a system to synchronize pages and the blog with pages in Mediawiki (so that you can edit the wiki and the content is shown on the Diem website) too, and this apparently not so easy task was just smooth because, hey, you have symfony there and you can write tasks using doctrine records and so on.

So, given that I was a total newbie at Diem and I could do everything I wanted without much pain (less than expected for a new platform), and that the base (layout, editor, admin panel, search engine, etc.) worked just fine, I am very glad.

PS: Given that I found that Diem suited my needs, I didn't try Sympal, so I cannot say anything about it. It was a matter of luck and perhaps eye candy to try Diem before Sympal, I don't have rational arguments for this choice. I will install it in the future and play a bit with it.

Changing colors of ls

2010-05-05T16:55:49+02:00

I have been quite a long time living with files with 777 permission being displayed in blue over green background whenever i do ls, which makes text quite illegible. Sometimes you don't see a problem even when it is in front of you. Well, I took the time to fix it today. Here is the solution:

dircolors --print-database &> ~/.dircolors

It will create a .dircolors file with the color specification vim .dircolors. Now we edit this file and change this line:

OTHER_WRITABLE 34;42 # dir that is other-writable (o+w) and not sticky

for this one:

OTHER_WRITABLE 01;37;42 # dir that is other-writable (o+w) and not sticky

That will tell ls to display these files with white bold letters over green background, mucho more legible.

In case you prefer different colors, here is what these codes mean:

# Attribute codes:
# 00=none 01=bold 04=underscore 05=blink 07=reverse 08=concealed
# Text color codes:
# 30=black 31=red 32=green 33=yellow 34=blue 35=magenta 36=cyan 37=white
# Background color codes:
# 40=black 41=red 42=green 43=yellow 44=blue 45=magenta 46=cyan 47=white

Last step is to edit ~/.bashrc to make dircolors load these colors each time a new term is opened. Open .bashrc and add these lines:

if [ -x /usr/bin/dircolors ]; then
 d=.dircolors
 test -r $d && eval "$(dircolors $d)" || eval "`dircolors -b`"
fi

Open a new terminal to see the changes o just run

source .bashrc

If you want to see the changes applied in the current terminal.

A workflow with remote branches on git

2010-03-12T01:09:50+01:00

This is the workflow that we are currently using for work with remote branches.

Pushing local branch to remote

git push origin my-branch:refs/heads/my-branch
git config branch.my-branch.remote origin
git config branch.my-branch.merge refs/heads/my-branch

That will create a new branch in the remote repository and the appropriate ref.

Pulling from remote to local branch

Before checking out, we need to fetch the branch, because our repo is not aware of it. Run this to understand what I mean.:

git remote show origin

So... let's fetch and then we'll be able to do the checkout.

git fetch
git checkout -b my-branch origin/my-branch

You can add a --track to the last command to set up upstream configuration (see the docs)

Grabar lo que está sonando (con pulseaudio)

2010-02-24T14:17:33+01:00

A veces interesa grabar en un fichero lo que está sonoando. Por ejemplo, yo me grabo una sesión de spotify para oirla cuando voy a correr. Aquí está el cómo hacerlo en Linux con pulseaudio: Encontré este script que explicaba cómo hacerlo, pero no me iba por tener el sistema en español. Un par de cambios tontos en el script y queda:

#!/bin/bash
WAV="$1"
if [ -z "$WAV" ]; then
 echo "Uso: $0 OUTPUT.WAV" >&2
 exit 1
fi
rm -f "$WAV"

# Get sink monitor:
MONITOR=$(pactl list | grep -A1 '^\*\*\* Origen #' | \
 grep '^Nombre: .*\.monitor$' | cut -d" " -f2 | tail -n1)

# Record it raw, and convert to a wav
echo "Grabando a $WAV ..."
echo "Cierra esta ventana para terminar"
parec -d "$MONITOR" | sox -t raw -r 44k -sLb 16 -c 2 - "$WAV"

Lo graba en un wav que luego puede uno pasar a mp3 o a lo que quiera.

Forcing a javascript file to be loaded at the end in symfony

2010-02-23T19:18:15+01:00

Very often, we want in symfony 1.x javascript files to be loaded after another. For instance, you may your files containing jquery functions to be loaded after the main jquery.js file to avoid the typical error:

$ is not defined

use_javascript (and addJavascript) accepts a 'position' parameter to deal with this. Position can be 'first' or 'last'. For instance:

<?php use_javascript('file-you-want-to-load-at-the-end.js', 'last')?>

Mechanize force encoding error

2010-01-17T00:38:25+01:00

Ruby's mechanize gem is very useful, specially when combined with Nokogiri. But if you find an encoding error like:

/var/lib/gems/1.8/gems/mechanize-0.9.3/lib/www/mechanize/page.rb:37:in `initialize': undefined method `force_encoding' for <String:0xb7410070> (NoMethodError)

Then you will need to apply this patch ;)

http://github.com/eric/mechanize/commit/7fd877c60cbb3855652c390c980df1dedfaed820

Gzipping js and css files

2009-12-03T04:08:49+01:00

This week we are optimizing Makoondi to make it load faster. I am using the page speed plugin by Google. One of the first advices it gives is Enable gzip compression, Compressing the following resources with gzip could reduce their transfer size by about two thirds.

I got it working pretty eadily for the .html files, just by adding this to http.conf:

AddOutputFilterByType DEFLATE text/css text/html text/javascript application/x-javascript application/javascript

But it was not working for .js and .css files, no wonder how many filetypes I could add to the list. It took me a while to find the solution here by Sören Kuklau, and I am so happy that I will replicate it here :) I added this chunk of code to my http.conf:

<Directory /var/www/makoondi/web/js/>
    AddHandler application/x-httpd-php .js php_value auto_prepend_file gzip-js.php php_flag zlib.output_compression On
</Directory>
<Directory /var/www/makoondi/web/css/>
    AddHandler application/x-httpd-php .css php_value auto_prepend_file gzip-css.php php_flag <lib.output_compression On
</Directory>

Then in the .../web/css/ directory I put this simple php file called gzip-css.php:

<?php
       header("Content-type: text/css; charset: UTF-8");
?>

And in the .../web/css/ directory gzip-js.php:

<?php
        header("Content-type: text/javascript; charset: UTF-8");
?>

Compressing the following resources with gzip could reduce their transfer size by about two thirds

Datepicker in symfony with i18n

2009-11-23T03:13:39+01:00

Some time ago we decided in Makoondi that would be better to have one of these jquery calendars where the user picks the date:

The first version was easy to implement with sfFormExtraPlugin , that implements sfWidgetFormJQueryDate.

$this->setWidget('available_from', new sfWidgetFormJQueryDate(array(
                        'image'=>'/images/icons/calendar.gif',
                        'culture' => sfContext::getInstance()->getUser()->getCulture(),
                        'format' => '%day%/%month%/%year%')
                       ));

You will need, apart from the plugin, some jquery files that you can download from the jquery-ui website:

jquery-ui-1.7.2.custom.min.js
jquery.datePicker.js

and, in order to use the i18n, you will need the localized version of jpicker from this website. e. g. the file with translations to Spanish would be:

jquery.datepick-es.js

But be aware! you will need to change mentions to $.datepick to $datepicker in that file in order to get it working!

$.datepicker.regional['es'] = {
...
$.datepicker.setDefaults($.datepicker.regional['es']);

otherwise you will get a "$.datepick is undefined" error :(

CSRF tokens in symfony

2009-10-22T19:23:45+02:00

What is that CSRF thing?

CSRF, or Cross-site request forgery, is a form of attack that takes advantage on the trust that a web application has on the user’s browser. If a different site than yours makes a user do a request to your site, since the user has a cookie session for your site, if you don’t have further protection, your application will trust the user’s browser and execute the request. But the user is not doing it voluntarily.

Wikipedia explains it better, but basically if you are logged in into your electronic banking account and you visit another web, you won’t like that that page makes your browser perform a request to your bank to transfer your money to somebody else’’s account. And you don’t want to be the programmer of that electronic banking app ;).

How to prevent it? One way is CSRF tokens. This is the solution that symfony employs.

What is a CSRF token?

A CSRF token is a key generated by the server that depends on a secret key, a session id and the class of the form. You provide it with a form as a hidden value, and you expect the browser to send that key back to you when the form is submitted. As the attacker doesn’t know your secret key, he cannot generate a valid token. Simple, uh?

Let’s take a look at the relevant pieces of code of sfForm:

$CSRFProtection    = false,
$CSRFSecret        = null,
$CSRFFieldName     = '_csrf_token',

The first variable determines whether the CSRF protection is activated. CSRFSecret holds the secret key of theapplication, and CSRFFieldName is… the name of the field in the form.

/**
* Returns a CSRF token, given a secret.
*
* If you want to change the algorithm used to compute the token, you
* can override this method.
*
* @param string $secret The secret string to use (null to use the current secret)
*
* @return string A token string
*/
public function getCSRFToken($secret = null)
{
  if (is_null($secret))
  {
    $secret = self::$CSRFSecret;
  }

  return md5($secret.session_id().get_class($this));
}

See? it is doing an MD5 encryption with your session_id, your secret and the name of the class of the form. The session is important, because if the form is left open enough time to let the session expire, the validation will fail.

/**
* Adds CSRF protection to the current form.
*
* @param string $secret The secret to use to compute the CSRF token
*/
public function addCSRFProtection($secret)
{
  if (false === $secret || (is_null($secret) && !self::$CSRFProtection))
  {
    return;
  }

  if (is_null($secret))
  {
    if (is_null(self::$CSRFSecret))
    {
      self::$CSRFSecret = md5(__FILE__.php_uname());
    }

 $secret = self::$CSRFSecret;
 }

 $token = $this->getCSRFToken($secret);

 $this->validatorSchema[self::$CSRFFieldName] = new sfValidatorCSRFToken(array('token' => $token));
 $this->widgetSchema[self::$CSRFFieldName] = new sfWidgetFormInputHidden();
 $this->setDefault(self::$CSRFFieldName, $token);
}

This function sets a secret key if there is not $secret parameter depending on the uname of the machine, but the interesting thing here is that the validator for the field is set to sfValidatorCSRFToken:

class sfValidatorCSRFToken extends sfValidatorBase
{
  /**
  * @see sfValidatorBase
  */
  protected function configure($options = array(), $messages = array())
  {
    $this->addRequiredOption('token');

    $this->setOption('required', true);

    $this->addMessage('csrf_attack', 'CSRF attack detected.');
  }

  /**
  * @see sfValidatorBase
  */
  protected function doClean($value)
  {
    if ($value != $this->getOption('token'))
    {
      throw new sfValidatorError($this, 'csrf_attack');
    }

  return $value;
  }
}

Formularios empotrados dinámicos en Symfony

2009-10-12T19:21:15+02:00

Los formularios embedded de symfony 1.2 son una herramienta muy útil, pero ¿cómo agregarlos dinámicamente desde la vista con AJAX?

Supongamos una aplicación simple. Se trata de tarjetas con una pregunta y su respuesta.

Podemos querer poner una imagen como pista, empotrando un formulario PictureForm:

class CardForm extends BaseCardForm
{
  public function configure()
  {
    unset($this['cardset_id'],$this['usercard_list']);
    $this->embedForm('picture', new PictureForm());

  }
}

y quedaría así:

Luego esto con suerte lo maquetará otra persona ;)

¿Pero y si nos interesa que pueda haber más de una imagen, en número variable?

Pues en ese caso nos tenemos que arremangar y meter más código. Empezamos por el formulario (CardForm.class.php en mi caso):

class CardForm extends BaseCardForm
{
  public function configure()
  {
    unset($this['cardset_id'],$this['usercard_list']);

    //Empotramos al menos un formulario de pictures
    $pictures = $this->getObject()->getPictures();
    if (!$pictures){
      $picture = new Picture();
      $picture->setCard($this->getObject());
      $pictures = array($picture);
    }

    //Un formulario vacío hará de contenedor para todas las pictures
    $pictures_forms = new SfForm();
    $count = 0;
    foreach ($pictures as $picture) {
    $pic_form = new PictureForm($picture);
    //Empotramos cada formulario en el contenedor
    $pictures_forms->embedForm($count, $pic_form);
    $count ++;
  }
  //Empotramos el contenedor en el formulario principal
  $this->embedForm('pictures', $pictures_forms);

  }
}

Este código nos valdrá tanto para objetos nuevos (en ese caso empotramos un solo formulario de pictures para empezar), como para objetos con pictures ya empotradas.

Ahora vamos a hacer en el mismo fichero una función AddPicture() que nos será muy útil:

public function addPicture($num){
  $pic = new Picture();
  $pic->setCard($this->getObject());
  $pic_form = new PictureForm($pic);

  //Empotramos la nueva pícture en el contenedor
  $this->embeddedForms['pictures']->embedForm($num, $pic_form);
  //Volvemos a empotrar el contenedor
  $this->embedForm('pictures', $this->embeddedForms['pictures']);
}

Esta función añade un PictureForm a al formulario. Esto nos va a venir bien por dos razones (si esto no se entiende bien ahora, no importa, luego lo veremos con detalle):

Al añadir con AJAX una picture, llamaremos a un action que ejecutará esta función para obtener un formulario con el número de pictures empotradas adecuado. Entonces el action renderizará solo la que nos interesa. Y esto será lo que insertaremos en la vista.
Al asociar (bind) el formulario, necesitamos que el formulario que va asociarse a los datos que enviemos tenga el número correcto de pictures ya empotradas. Es decir, si vamos a asociar un formulario que ha introducido el usuario con un número determinado de pictures empotradas, el objeto formulario con el que vamos a hacer bind debe tener el mismo número de formularios empotrados. Para eso, sobreescribiremos la función bind para que añada tantas pictures empotradas como nos haga falta.

Vamos por pasos. El primero. En la vista, añadimos esta función de AJAX. Yo uso Jquery, si usas Prototype u otra cosa no te será difícil hacer algo parecido:

<script type=&quot;text/javascript&quot;>
var pics = <?php print_r($form['pictures']->count())?>;

function addPic(num) {
  var r = $.ajax({
    type: 'GET',
    url: '<?php echo url_for('card/addPicForm')?>'+'<?php echo ($form->getObject()->isNew()?'':'?id='.$form->getObject()->getId()).($form->getObject()->isNew()?'?num=':'&amp;num=')?>'+num,
    async: false
  }).responseText;
  return r;
}
$().ready(function() {
  $('button#add_picture').click(function() {
    $(&quot;#extrapictures&quot;).append(addPic(pics));
    pics = pics + 1;
  });
});
</script>

y tras el formulario un botón para añadir subformularios y un espacio para colocarlos cuando nos los la función AJAX.

<?php echo $form ?>
<div id=&quot;extrapictures&quot;/>

<tr><td><div><button id="add_picture" type="button"><?php echo "Añadir otra imagen"?></button></div></td></tr>

En el action, hacemos la función executeAddPicForm()

public function executeAddPicForm($request)
{
 $this->forward404unless($request->isXmlHttpRequest());
 $number = intval($request->getParameter("num"));

 if($card = CardPeer::retrieveByPk($request->getParameter('id'))){
 $form = new CardForm($card);
 }else{
 $form = new CardForm(null);
 }

 $form->addPicture($number);

 return $this->renderPartial('addPic',array('form' => $form, 'num' => $number));

}

Que básicamente lo que hace crear un formulario y llamar al addPicture() de ese formulario para tener un nuevo formulario con el número de elementos empotrados que nos interesa. Entonces le pasamos a una template addPic ese objeto formulario y el número del subformulario empotrado que queremos representar. La template tiene este código:

<?php echo $form['pictures'][$num]['id']->render()?>
<?php echo $form['pictures'][$num]['file']->renderRow();?>

Con esto ya tenemos un botón que añade nuevos subformularios. Pero ¿y a la hora de guardar? Como hemos dicho antes, necesitaremos que bind prepare el formulario. El nuevo código de bind será:

public function bind(array $taintedValues = null, array $taintedFiles = null)
{
  foreach($taintedValues['pictures'] as $key=>$newPic)
  {
    if (!isset($this['pictures'][$key]))
    {
      $this->addPicture($key);
    }
  }

  parent::bind($taintedValues, $taintedFiles);
}

Y con esto ya podemos usar este formulario como uno cualquiera. No es lo más cómodo del mundo, pero podemos seguir aprovechando sfForm totalmente en lugar de escribir toda la lógica de tratamiento de formulario nosotros mismos.

Solo nos quedará el trabajo de maquetación, descomponiendo campos de formularios para que todo quede bien :)

Dynamic embedded forms in symfony

2009-10-12T19:17:03+02:00

Embedded forms in symfony are a very useful tool, but ¿how to add them dynamically from the view using AJAX?

Simple form

Let’s take a simple applicaton. There will be flashcards with a question and its answer. The form for creating these cards could be this:

With an embedded form

We may want to offer an image as a hint, embedding a PictureForm:

class CardForm extends BaseCardForm
{
  public function configure()
  {
    unset($this['cardset_id'],$this['usercard_list']);
    $this->embedForm('picture', new PictureForm());
  }
}

Dynamic embedded forms

¿What if we are interested in having more than one image as hint, a variable number of them?

In this case we need to write more code. Let’s start with the form (CardForm.class.php in my case):

class CardForm extends BaseCardForm
{
  public function configure()
  {
    unset($this['cardset_id'],$this['usercard_list']);

    //Embedding at least a form
    $pictures = $this->getObject()->getPictures();
    if (!$pictures){
      $picture = new Picture();
      $picture->setCard($this->getObject());
      $pictures = array($picture);
    }

    //An empty form will act as a container for all the pictures
    $pictures_forms = new SfForm();
    $count = 0;
    foreach ($pictures as $picture) {
      $pic_form = new PictureForm($picture);
      //Embedding each form in the container
      $pictures_forms->embedForm($count, $pic_form);
      $count ++;
    }
    //Embedding the container in the main form
    $this->embedForm('pictures', $pictures_forms);
  }
}

This code will be valid for new objects (we will embed only a form to start with) and for objects with already embedded pictures.

Now we will write an AddPicture() function that will be very useful:

public function addPicture($num){
  $pic = new Picture();
  $pic->setCard($this->getObject());
  $pic_form = new PictureForm($pic);

  //Embedding the new picture in the container
  $this->embeddedForms['pictures']->embedForm($num, $pic_form);
  //Re-embedding the container
  $this->embedForm('pictures', $this->embeddedForms['pictures']);
}

This function adds a PictureForm to the main form. This is useful for two reasons (we will see this in more detail later):

When adding a picture from an AJAX call, we will call an action that will run this function to obtain a form with the embedded form . Them, that action will render only what we need. And that will be what we will inject in the view.
When we bind this form, we need the form that is going to be bound to the user-written data to have the right number of embedded pictures. That is, if we are going to bind a user-introduced form with a given number of embedded pictures, the form object must have the same number of embedded forms. For that reason, we will overload the bind function to add as many embedded pictures as we need.

Step by step. First: in the view, we will add this AJAX function. I use Jquery, it is not difficult to write the same thing in Prototype:

<script type="text/javascript">
var pics = <?php print_r($form['pictures']->count())?>;

function addPic(num) {
  var r = $.ajax({
    type: 'GET',
    url: '<?php echo url_for('card/addPicForm')?>'+'<?php echo   ($form->getObject()->isNew()?'':'?id='.$form->getObject()->getId()).($form->getObject()->isNew()?'?num=':'&num=')?>'+num,
    async: false
  }).responseText;
  return r;
}
$().ready(function() {
  $('button#add_picture').click(function() {
    $("#extrapictures").append(addPic(pics));
    pics = pics + 1;
  });
});
</script>

And after the form a button to add forms and a place to inject them after the AJAX call:

<?php echo $form ?>
<div id="extrapictures"/>

<tr><td><div><button id="add_picture" type="button"><?php echo "Añadir otra imagen"?></button></div></td></tr>

In the action, we write the this executeAddPicForm() function:

public function executeAddPicForm($request)
{
  $this->forward404unless($request->isXmlHttpRequest());
  $number = intval($request->getParameter("num"));

  if($card = CardPeer::retrieveByPk($request->getParameter('id'))){
    $form = new CardForm($card);
  }else{
    $form = new CardForm(null);
  }

  $form->addPicture($number);

  return $this->renderPartial('addPic',array('form' => $form, 'num' => $number));
}

Basically, this function creates a form and calls to addPicture() to have a new form with an embedded form withh the right number. Then we run a partial addPic with this form and the number of embedded form to show. This partial has the following code:

<?php echo $form['pictures'][$num]['id']->render()?>
<?php echo $form['pictures'][$num]['file']->renderRow();?>

With this we have a button that adds new subforms. But, what about saving? As we said before, we will need bid to do part of the task. Its new code will be:

public function bind(array $taintedValues = null, array $taintedFiles = null)
{
  foreach($taintedValues['pictures'] as $key=>$newPic)
  {
    if (!isset($this['pictures'][$key]))
    {
      $this->addPicture($key);
    }
  }
  parent::bind($taintedValues, $taintedFiles);
}

And with this change we can use this form as a normal one.

PS: I must give people credit about this one. I looked at several places for this info and this post was very useful. It is basically the same, but more focused in adding fields synamically instead of embedded forms.

PS2: See also this post and this one about this topic.

Heisenberg uncertainty principle in e-mail campaigns analytics

2009-10-12T18:28:46+02:00

Heisenberg uncertainty principle in e-mail campaigns analytics

We have been experiencing something quite paradoxical with our e-mail campaigns and Google Analytics. To track an e-mail campagin with GA you need to append some long parameters to the URLs, like

http://es.makoondi.com/ofertas_habitacion/new?utm_campaign=newoffer&utm_medium=email&amp;utm_source=pcom

for the base URL:

http://es.makoondi.com/ofertas_habitacion/new

And, yes, we could track this campaign. But we noticed that our campaign was less effective since we started measuring it. It looks like our target doesn’t like so long URLs.

Then we tried to use bit.ly, to shorten the URL:

http://bit.ly/rEl0r

And the result was terrible. People like these cryptic urls even less than the long ones.

So, we decided that we couldn’t mesure the effectivity of the campaign without altering it (in the plain text emails). We aren’t measuring it anymore, but the results are better. And since that is what counts…

Error in multipart forms in symfony

2009-10-09T19:12:49+02:00

When $form->bind() throws this error:

This form is multipart, which means you need to supply a files array as the bind() method second argument.

It means that at least one field in the form is multipart. That means that the form contains files. That’s why bind needs a second argument, usually with the form $request->getFiles($form->getName());

    $this->form->bind($request->getParameter($this->form->getName()), $request->getFiles($this->form->getName()));

Error de formulario multipart en symfony

2009-10-09T19:11:26+02:00

Cuando al hacer un $form->bind() nos da el siguiente error:

This form is multipart, which means you need to supply a files array as the bind() method second argument.

Significa que alguno de los campos del formulario es multipart. Es decir, que el formulario contiene campos de fichero. Por eso bind necesita un segundo argumento, que suele ser $request->getFiles($form->getName());

$this->form->bind($request->getParameter($this->form->getName()), $request->getFiles($this->form->getName()));

Para más información, en una entrada pasada expliqué en profundiad qué iba haciendo bind().

Dentro de $form->bind() de symfony

2009-09-30T19:14:26+02:00

La documentación de formularios de Symfony va mejorando, pero sigue habiendo muchos puntos oscuros. Los formularios siguen siendo una de partes más complejas de entender de Symfony. Hay una forma simple para hacer cosas simples, pero a la que te quieres mover de ahí, es necesario entender cómo funciona todo por dentro.

Y el punto más importante del proceso está en la función bind(). Por ejemplo, si hemos hecho un post validator y no funciona, si estamos trabajando con formularios empotrados (embedded), o simplemente si algo no funciona, necesitaremos entender qué está pasando en este método. Así pues, ¿Qué hace bind()? Según la documentación de formularios de Symfony 1.2 bind() hace lo siguiente:

Cuando el formulario se asocia (bind) a los datos externos usando bind(), el formulario pasa al estado «bound» (ya veremos que esto no significa gran cosa) y se ejecutan las acciones siguientes:

Se ejecuta el proceso de validación
Los mensajes de error se almacenan en el formulario para que estén disponibles para la template
Los valores por defecto del formulario son reemplazados por los datos que ha introducido el usuario.

Vamos a verlo con más detalle. Una función de procesado de formulario típica (bind(), y si es válido, save()) tiene esta forma:

protected function processForm(sfWebRequest $request, sfForm $form)
{
  $form->bind($request->getParameter($form->getName()), $request->getFiles($form->getName()));
  if ($form->isValid())
  {
    $card = $form->save();
    $this->redirect('card/edit?id='.$card->getId());
  }
}

Esta operación tiene dos actores: la $request y el $form. Dentro de todo lo que hay en una $request, si todo ha ido bien habrá un array. En el caso de este objeto (card), nuestra $request contiene este campo:

[postParameters:protected] => Array
        (
            [card] => Array
                (
                    [question] => elefante
                    [answer] => mamífero
                    [id] =>
                    [_csrf_token] => a288d0365a6bb4aab5a11789534a4ddb
                )
        )

Así que tenemos los datos del formulario en un array. Este array es el que enviamos a bind en el primer parámetro al hacer

$form->bind($request->getParameter($form->getName()), $request->getFiles($form->getName()));

¿Qué hace bind con esto? Miremos el código de sfForm->bind() :

 /*
 * Binds the form with input values.
 *
 * It triggers the validator schema validation.
 *
 * @param array $taintedValues An array of input values
 * @param array $taintedFiles  An array of uploaded files (in the $_FILES or $_GET format)
 */
public function bind(array $taintedValues = null, array $taintedFiles = null)
{
  $this->taintedValues = $taintedValues;
  $this->taintedFiles  = $taintedFiles;
  $this->isBound = true;
  $this->resetFormFields();

  if (is_null($this->taintedValues))
  {
    $this->taintedValues = array();
  }

  if (is_null($this->taintedFiles))
  {
    if ($this->isMultipart())
    {
      throw new InvalidArgumentException('This form is multipart, which means you need to supply a files array as the bind() method second argument.');
    }

    $this->taintedFiles = array();
  }

 try
  {
    $this->values = $this->validatorSchema->clean(self::deepArrayUnion($this->taintedValues, self::convertFileInformation($this->taintedFiles)));
    $this->errorSchema = new sfValidatorErrorSchema($this->validatorSchema);

    // remove CSRF token
    unset($this->values[self::$CSRFFieldName]);
  }
  catch (sfValidatorErrorSchema $e)
  {
    $this->values = array();
    $this->errorSchema = $e;
  }
}

En las dos primeras líneas guarda nuestro array de valores que viene de la request en variables del objeto. La siguiente marca el formulario como bound. Es decir, que aunque haya un desastre en un punto posterior del código de bind(), si llamamos a $form->isBound() nos dirá que efectivamente está «bound». Es decir, que isBound() significa «se ha llamado a bind para este formulario» y no «el bind ha funcionado». resetFormFields() simplemente vacía el array de $this->formFields y también $form->formFieldSchema . El fieldschema es el objeto que realiza el render de un field del formulario cuando lo queremos representar.

Lo siguiente que hace es comprobar que taintedValues y taintedFiles existan y sean coherentes con el formulario (si es un formulario multipart, necesitará taintedFiles).

Tras los preparativos ejecuta una línea que es la que hace los tres puntos de la descripción de bind(). Es decir, la que hace lo que esperamos de bind():

$this->values = $this->validatorSchema->clean(self::deepArrayUnion($this->taintedValues, self::convertFileInformation($this->taintedFiles)));

De dentro a afuera:

convertFileInformation() «convierte un array de fichero a un formato que cumple la convención de nombres de $_GET y $_POST».
deepArrayUnion() es una función recursiva que fusiona dos arrays.
Y este array es el que se le pasa a $this->validatorSchema->clean().

Es decir, que le pasa el muerto al validatorSchema, que es el que carga con el peso del bind. Una conclusión que podemos sacar es que, si hay formularios embedded, sus métodos bind no se van a llamar, pero sí sus validadores. Así que la lógica de sus binds tiene que ir en alguna otra parte.

¿Qué hace $this->validatorSchema->clean()? llama a doClean() que hace esto:

protected function doClean($values)
{
  if (is_null($values))
  {
    $values = array();
  }

  if (!is_array($values))
  {
    throw new InvalidArgumentException('You must pass an array parameter to the clean() method');
  }

  $clean  = array();
  $unused = array_keys($this->fields);
  $errorSchema = new sfValidatorErrorSchema($this);

  // check that post_max_size has not been reached
  if (isset($_SERVER['CONTENT_LENGTH']) &amp;amp;&amp;amp; (int) $_SERVER['CONTENT_LENGTH'] > $this->getBytes(ini_get('post_max_size')))
  {
    $errorSchema->addError(new sfValidatorError($this, 'post_max_size'));

    throw $errorSchema;
  }

  // pre validator
  try
  {
    $this->preClean($values);
  }
  catch (sfValidatorErrorSchema $e)
  {
    $errorSchema->addErrors($e);
  }
  catch (sfValidatorError $e)
  {
    $errorSchema->addError($e);
  }
  // validate given values
  foreach ($values as $name => $value)
  {
    // field exists in our schema?
    if (!array_key_exists($name, $this->fields))
    {
      if (!$this->options['allow_extra_fields'])
      {
        $errorSchema->addError(new sfValidatorError($this, 'extra_fields', array('field' => $name)));
      }
      else if (!$this->options['filter_extra_fields'])
      {
        $clean[$name] = $value;
      }

      continue;
    }

    unset($unused[array_search($name, $unused, true)]);

    // validate value
    try
    {
      $clean[$name] = $this->fields[$name]->clean($value);
    }
    catch (sfValidatorError $e)
    {
      $clean[$name] = null;

      $errorSchema->addError($e, (string) $name);
    }
  }
  // are non given values required?
  foreach ($unused as $name)
  {
    // validate value
    try
    {
      $clean[$name] = $this->fields[$name]->clean(null);
    }
    catch (sfValidatorError $e)
    {
      $clean[$name] = null;

      $errorSchema->addError($e, (string) $name);
    }
  }

  // post validator
  try
  {
    $clean = $this->postClean($clean);
  }
  catch (sfValidatorErrorSchema $e)
  {
    $errorSchema->addErrors($e);
  }
  catch (sfValidatorError $e)
  {
    $errorSchema->addError($e);
  }

  if (count($errorSchema))
  {
    throw $errorSchema;
  }
  return $clean;
}

Y aquí está el quid de la cuestión. Tras asegurarse de que los datos de entrada tienen sentido, llama al preValidator, si es que hay uno definido, en preClean(). Entonces llama a la validación campo a campo. Y acaba llamando al postValidator, en postClean(). Hay que fijarse en cosas como que nuestro postValidator realmente devuelva valores, porque si no devuelve nada, si nos fijamos en el código veremos que no retornaremos los valores «cleaned» y estaramos liándola, porque entonces $this->values del formulario quedará vacío.

Resulta complicado encontrar errores así. Por eso es interesante entender qué está pasando al hacer un bind. En este post he tratado de resumirlo y de reunir el código que hay disperso por la API, para que uno pueda tenerlo a mano y entender qué puede estar pasando cuando las cosas fallan.