lundi 25 juillet 2016

AbstractMethodError when using traits in Groovy/Grails

Introduction

The 2.3 version of groovy introduced traits, sets of fields and methods that can be implemented by classes like superpowered interfaces or abstract classes; they not only contain the declaration, but also the definition of methods and can contain attributes. A class can implement multiple traits at once, providing some kind of multiple inheritance for groovy that classes could not provide in java.

AbstractMethodError

This feature being quite new, the compiler still has some glitches when dealing with traits. The most notable error is that when you change the source of a trait, it won't recompile the classes implementing it, causing sometimes an AbstractMethodError at runtime when the class doesn't find a recently added or modified method of the trait.

Resolution

To force the recompilation of the sources files, you can either do a full clean and rebuild of your project - but it can be a huge loss of time for a large project - or add a trivial modification to each source file implementing the trait to trigger their recompilation instead of using the cache; adding a comment for example, then compile - but for a trait that is widely used, like an helper trait used in every unit test, adding a comment to a large number of file can be tedious, and you must remember to remove them before committing...

A better solution

A better solution is to use a script add these comments in all the files of a folder to trigger their rebuilding and another one to remove the comments. This way you avoid the manual modification of the files while still avoiding the rebuild the full project.

The workflow after getting an AbstractMethodError:

  1. Run the mark_files_as_changed.py script on the source folder that contains the classes implementing the trait. For a test helper trait use it on the test/ folder, for a controller helper trait run it on the controllers/ folder.
  2. Recompile you project. Only the 'changed' classes, that have been marked by your script will be recompiled.
  3. Run the unmark_files.py on the same folder, removing the comments added by the first script  to clean the files.

Implementation in Python:

mark_files_as_changed.py:

"""
This script appends comments to all files in the folder whose path is given as parameter, to ensure that
Intellij IDEA rebuilds them. This is useful to ensure that classes that implements a trait are recompiled when that trait is edited.
After compilation, please run the unmark files to remove the added comments.
Created on Tue Jul 19 14:04:51 2016
@author: hschoonjans
Python 3
"""

import os
import sys

walk_dir = sys.argv[1]

print('walk_dir = ' + walk_dir)

print('walk_dir (absolute) = ' + os.path.abspath(walk_dir))

APPEND = 'a'

for root, subdirs, files in os.walk(walk_dir):
    print('--\nroot = ' + root)

    for filename in files:
        file_path = os.path.join(root, filename)

        print('\t- file %s (full path: %s)' % (filename, file_path))

        with open(file_path, APPEND) as file:
            file.write("//CHANGED")


unmark_files.py:


"""
See mark_file_as_changed.py.
Removes the "//CHANGED" comments in all the files of the folder.
Created on Tue Jul 19 14:04:51 2016
@author: hschoonjans
Python 3
"""

import os
import sys

walk_dir = sys.argv[1]

print('walk_dir = ' + walk_dir)

READ_MODE = 'r'
WRITE_MODE = 'w'

for root, subdirs, files in os.walk(walk_dir):
    print('--\nroot = ' + root)

    for filename in files:
        file_path = os.path.join(root, filename)

        print('\t- file %s (full path: %s)' % (filename, file_path))

        f = open(file_path, READ_MODE)
        file_data = f.read()
        f.close()

        new_data = file_data.replace("//CHANGED", "")

        f = open(file_path, WRITE_MODE)
        f.write(new_data)
        f.close()


You can also find the implementation on my GitHub.


jeudi 14 juillet 2016

Practical guide to neural style with docker

Google has recently made some noises with it's amazing neural dream images, images generated by neural networks. Another popular use of neural networks for image processing is called "neural style", the use of neural algorithms to apply the style of an image to another.

I'll explain here how to quickly use neural style algorithms to transform your photos. I'll assume you aren't interested into the theory behind the method and just want to get started quickly. To avoid a cumbersome installation and possible incompatibilities of the technologies used with your machine, I'll make you use a virtual machine called Docker and download an image with all the needed libraries already installed.

First, you'll need to have Docker installed on your computer. I'll assume that you will manage to do it by youself.

With docker installed, pull the image containing the code:
docker pull heschoon/neural-style

It will take some time, as you are downloading a 1Gb virtual machine. While it downloads, collect at least two images in one folder on your computer. We will together learn how to apply the style of the first image ( let's call it style.png ) to the second image ( content.png ).

Tip: choose as style.jpg an image with a memorable style!

A popular style image is the starry night painting of Van Gogh:


In this tutorial, I'll use the picture of an handsome programmer as content.jpg.


Put both image in the same folder on your computer. Now you can run the virtual machine:
docker run -it -v <path/to/your/images/folder>:/root/input-images/ \
heschoon/neural-style /bin/bash

In case you wonder what' does the command: "docker run" will run the virtual machine, "-it" attaches an interactive terminal on your virtual machine, "-v <path/to/your/images/folder>input-images" mounts the folder containing the images on the virtual machine, "heschoon/neural-style" is the name of the virtual machine to run. This docker image is based on the jcjohnson/neural-style implementation of the neural-style algorihtm.

You can now move to the folder that contains the code:
cd ~/neural-style/

Ensure that the image folder has been successfully mounted on your docker:
ls ../input-images

If the command list the names of the images you have in your folder; probably style.jpg and content.jpg, the folder has been successfully mounted.

You can generate a new image:
th neural_style.lua -gpu -1 -style_image ../input-images/style.jpg \
-content_image ../input-images/content.jpg -image_size 256

The algorithm is now running. It my laptop it took 2 hours, so be patient.You might want to give more RAM to Docker if the algorithms crashes.

When the wait is over, copy the generated images into the shared folder, to make them persistent and accessible from your computer:
cp out* ../input-images

You can now exit the virtual machine by type the exit command, and look into your image folder. The generated output images should be available.



Now, you can try with other style and content images, and play with the other flags and commands described on the code repository.