/dev/blah

things i want to remember, things i want to share

Développeur Python et adepte Linux depuis 2005, Core développeur Kivy, passionné par beaucoup trop de choses.
Profil Github gpg signature bitcoin address

Entries tagged “python”

Kivy hackathon in paris

written by tshirtman, on 4/16/13 12:04 AM.

Last week saw the first 24h hackathon event entirely dedicated to Kivy happening, in Paris. It was initiated by a hightly motivated student of the Cifacom school in paris, David Scheck, and attended by 4 groups of students, each with 3 to 5 members. I was pleased to share the hard task to help students discover the technology with my colleague Julien Miotte, from Majerti during the event.

The student weren’t very familiar with Python, and totally new to kivy, so it was a really challenging situation for both the student and the framework, would they be able to achieve anything in the hard limit of 24 hours?

First, after the students brainstormed on their project ideas, we got them through the major concepts and important classes of Kivy, using the Getting started page, it’s hard to get the point of properties accross in 10 minutes to student that didn’t have experienced the need of them, but I tried to at least make them aware it would be useful. The kv language presentation at this point, even if rudimentary, was probably useful, as we will see later.

Then students went on to try to build their project, except for a few quickly solved issues with windows installation, the starting was smooth. Julien and I gave some more explanations about how to start building a basic App (the doc is there, but who reads the doc? ;)) and people started designing their interface, and build the core of their application.

Eat Smart

One of the most required feature was certainly the use of multiple screens in an application, to build menu, so the use of ScreenManager was explained to at least two groups. Most of the end results consisted of interfaces done mainly in Kv, and some internal logic to display data, most group had a quite ambitious target, using geolocalisation, databases, complex interactions, and I wasn’t very optimistic on the odds of seeing them completed. Although I was right in that, I was still happy of the good designs they came up with, applications were incomplete (to a notable exception, will come back to that soon), but some interfaces were beautiful, reactive and engaging, so it was nice to see the magic happen in such a short time.

Floor2be

One group had a very different objective of the others, and although they had a difficult start, it turns out they were probably the best at teamwork, and had chosen a target they could actually achieve, a game! These motivated coders/artists came up with a classic 2D sidescroller, when the main character has to avoid elements on her way. Simple, to the point, and a quite a good realisation in such a short time, the fact that they created original artworks and integrated them in their game, and that the general theme had a nice touch of humour in the current political events in France certainly hearned them points, on top of having a completed project. They even could test it on Android during the competition time, and it was running perfectly. Their hard work earned them the right to run in a bigger international competition in the near future the angel hack in paris, next month, congratulation! I can only hope they’ll chose kivy as their secret weapon in this competion too.

The BoutinRun team at work

I would love to have better pictures, but i didn’t think much of taking them during the event, I should have better ones soon.

Debugging python-for-android with gdb

written by tshirtman, on 11/13/12 1:55 AM.

So i’ve been sinking a lot of time in trying to make python-for-android support multiprocessing, and for various reasons, it turns out to be a bad idea®, but in doing that, i had to go to great extents to understand more about python-for-android, about gdb, and about android itself. So, if it can save some trouble to others, i’ll try to document here the things i found how to do in this situation.

So the first thing to do is to get python-for-android compiled with debug symbols, so you can feed gdb with them, i found android-ndk-r8b to work much better after (well, to work at all), so it’s good to start with it, i put my changes in a branch of python-for-android repository, aptly called `debug` but they are not that big in fact, it’s mostly removing stripping in places, and adding -g in others.

With those changes, one can build a python-for-android distribution usable for debugging.

./distribute.sh -f -m 'kivy' -d debug

then, build your to-be-debugged application with this distribution, and install it, from then, you need a shell access to your android terminal (i use ssh, because adb over wifi is flacky but that’s another story, adb shell is good), you must then check if your terminal has the gdbserver command, if not, you can push it from the ndk to the terminal, and give it execution rights.

adb push ~/android-ndk-r8b/prebuilt/android-arm/gdbserver
adb shell
chmod 755 gdbserver

now, you can start your test program (from the UI), and tell gdbserver to attach to it,

ps | grep python
gdbserver :5050 --attach $pid

obviously using the pid found using the command, i use a shortcut though

./gdbserver :5050 --attach $(ps | grep python | tail -n 1 | tr -s " " " "| cut -d" " -f2)

This will tell gdbserver to listen on port 5050 (you can chose about any number between 1024 and 65536) and then the fun part begins

using the gdb in the ndk, load the pymodules.so file

$ ~/android-ndk-r8b/toolchains/arm-linux-androideabi-4.4.3/prebuilt/linux-x86/bin/arm-linux-androideabi-gdb\
 ~/python-for-android/dist/debug/private/libpymodules.so

of course, this path (and the following ones) will need to be adapted to your system!

once in gdb, you’ll want to attach to the gdbserver on your device, i assume you are on the same wifi network as your device, but if not, you can do a port redirection through adb link i think

(gdb) target remote transformer:5050

If everything went fine, you should get a message a bit like this

Remote debugging using transformer:5051
warning: while parsing target library list (at line 2): No segment defined for com.showgen.processcraft.freemium:python
warning: Could not load shared library symbols for 87 libraries, e.g. /system/bin/linker.
Use the "info sharedlibrary" command to see the complete listing.
Do you need "set solib-search-path" or "set sysroot"?
warning: Unable to find dynamic linker breakpoint function.
GDB will be unable to debug shared library initializers
and track explicitly loaded dynamic code.
0x400b9384 in ?? ()

as the helpful message put it, we need to tell gdb where it can load the shared libs with debug symbols. Personnaly i found this setting of `solib-search-path`to work well:

(gdb) set solib-search-path \
/home/gabriel/python-for-android/src/obj/local/armeabi/:\
/home/gabriel/python-for-android/dist/debug/private/lib/python2.7/lib-dynload/:\
/home/gabriel/python-for-android/dist/debug/libs/armeabi/

You can check the result of this by using the command `info sharedlibrary` (or `i sh` for the lazy) to see if symbols have been found, the interresting part for me is:

0x56d31348  0x56d5c148  Yes         /home/gabriel/python-for-android/src/obj/local/armeabi/libsdl.so
0x579b5400  0x579d5798  Yes         /home/gabriel/python-for-android/src/obj/local/armeabi/libsdl_image.so
0x579fa428  0x57a2c1f8  Yes         /home/gabriel/python-for-android/src/obj/local/armeabi/libsdl_ttf.so
0x56a5a6b0  0x56a69c20  Yes         /home/gabriel/python-for-android/src/obj/local/armeabi/libsdl_mixer.so
0x5b985570  0x5bbc9e90  Yes         /home/gabriel/python-for-android/dist/billing/libs/armeabi/libpython2.7.so
0x56a79a1c  0x56a79c04  Yes         /home/gabriel/python-for-android/src/obj/local/armeabi/libapplication.so
0x56dee534  0x56dee5e8  Yes         /home/gabriel/python-for-android/src/obj/local/armeabi/libsdl_main.so
0x57604450  0x5761ed40  Yes         /home/gabriel/python-for-android/dist/billing/private/lib/python2.7/lib-dynload/_io.so
0x57a7426c  0x57a77630  Yes         /home/gabriel/python-for-android/dist/billing/private/lib/python2.7/lib-dynload/unicodedata.so
0x57b5b058  0x57b97d00  Yes         /home/gabriel/python-for-android/src/obj/local/armeabi/libsqlite3.so

Interrestingly, it seems the command is only useful after connecting to the target, doing it in reverse order doesn’t seem to bring any result, so the `search` is probably done only once, whet the search-patch is set.
Now, you can use the command `continue` or set breakpoints, or other rather common usage of gdb there. There are quite some resources out there about gdb usage :)

It’s not perfect, but i found it helping to be able to backtrace things on the tablet, in the nasty case you get a crash, or some weird “OSError”, with an unhelpful message, comming from the C side of things.

For the story, my error was likely caused by memory corruption due to the way we tried to reimplement the multiprocessing use of semaphore (sem_open being not implemented in android), and there is apparently no good way to do it. So i’ll look into android services instead when i have time, and try to give an API in python for that.

[Kivy] android-like pulldown menu

written by tshirtman, on 10/22/12 9:29 PM.

kivy all the things

As someone asked how to do such thing in kivy, i spent some time writting it, and as i (sadly) don’t blog on kivy often (if ever?), and since i think this example is quite telling about how kivy make such things quite easy, let’s talk a bit about this code.

To put things in context, what we want is the bar at the top of the android phones, that one can pull down to see notifications, this one is semi transparent and goes over the normal screen.

So, here is the code. first the kv part:

FloatLayout:
    FloatLayout:
        # placeholder for the "normal screen"
        Label:
            center: root.center
            text: 'test content'
            size_hint: None, None
            size: self.texture_size

    ScrollView:
        # container for the "notifications"
        y: dg.top # let's stick it to the top
        x: root.x # and to the left
        size_hint_y: None
        do_scroll_x: False # only vertical scrolling
        do_scroll_y: True
        
        # most of the magic is there, auto adjust size to follow the draggable label
        height: root.top - dg.y

        # let's put a nearly opaque black background
        canvas:
            Color:
                rgba: 0, 0, 0, .8
            Rectangle:
                pos: self.pos
                size: self.size
        
        # the actual notification container, with placeholder content
        BoxLayout:
            size_hint_y: None
            height: 1000
            orientation: 'vertical'
            Label:
                text: 'test'
            Label:
                text: 'test'
            Label:
                text: 'test'
            Label:
                text: 'test'
            Label:
                text: 'test'
            Label:
                text: 'test'
            Label:
                text: 'test'
            Label:
                text: 'test'
            Label:
                text: 'test'

    # the draggable label, which behaviour is defined in python file
    DraggableLabel:
        # some decoration behind the text
        canvas.before:
            Color:
                rgba: 0, 0, 0, 1
            Rectangle:
                pos: self.pos
                size: self.size
            Color:
                rgba: .5, .5, .5, 1
            Rectangle:
                pos: self.pos
                size: self.width, 1

        size_hint_y: None
        top: root.top
        # assign its id to "dg" so we can reference it elsewhere
        id: dg
        height: '20pt'
        text: 'drag me'

then the python part

from kivy.app import App 
from kivy.uix.label import Label
from kivy.animation import Animation


class DraggableLabel(Label):
    '''A label you can drag upside-down'''
    def on_touch_down(self, touch):
        if self.collide_point(*touch.pos):
            # assure ourselves we will get the updates of this motion
            touch.grab(self)
            return True

        return super(DraggableLabel, self).on_touch_down(touch)

    def on_touch_move(self, touch):
        if touch.grab_current is self:
            # really straightforward...
            self.y = touch.y
            return True

        return super(DraggableLabel, self).on_touch_move(touch)

    def on_touch_up(self, touch):
        if touch.grab_current is self:
            # check if the movement direction was up or down
            if touch.dy < 0:
                a = Animation(y=0) # down? put the bar all the way down
            else:
                a = Animation(top=self.parent.top) # up? put it at the top

            a.start(self) # actually start the animation
            return True

        return super(DraggableLabel, self).on_touch_up(touch)


class TabApp(App):
    pass


TabApp().run()

 

I think it doesn’t really need more explanations, the DraggableLabel is looking for touch events that are for it, first if they are on it, then if they are grabbed by itself, and move accordingly, and the kv auto adjust the size of the ScrollView to take all the distance between top of the screen and top of the DraggableLabel.

Of course, if things are not clear, feel free to ask questions :)

PyJNIus mailing list

written by tshirtman, on 9/6/12 3:43 PM.

Since there are some questions/discussions that doesn’t perfectly fit as github issues, i just created a google group to discuss PyJNIus development, i hope this will make collaboration easier.

https://groups.google.com/forum/#!forum/pyjnius-dev

pyjnius-dev@googlegroups.com

Pyjnius

written by tshirtman, on 8/20/12 4:46 PM.

J’ai bossé ces derniers jours avec Tito (oui, enfin, il a fait 90% du code ^^) sur une petite lib pour manipuler des objects java de façon transparente depuis python, à l’origine prévus pour le project python-for-android, mais c’est finalement utilisable sur desktop aussi, ça marche vraiment pas mal, et ça peut sans doute intéresser des gens, on a parfois besoin de bosser avec un système java depuis python, et cette lib fournis une interface je pense très agréable pour le faire.

Présentation sur le blog de Tito avec un peu plus de détails.

Le project est hébergé sur github.

Get it while it’s hot ! :)

Announcing PythonAR

written by tshirtman, on 8/7/12 9:56 PM.

Some time ago, doing heavy-client dev (with kivy, check it out, it’s great!), i got bored with the save/run/quit/edit workflow, and having used flask and other tools with “–reload” like options, i wondered if there was something more generalistic.

I didn’t find anything, thought i didn’t search a lot, but i decided to scratch that itch, and to build a solution usable for about anything.

So after a few hours of hacking, i got a first version of an app reloader, using the watchdog lib, and some subprocess hacking. However the application was stealing the focus everytime it was started, wich made it a bit less fun to use. Another problem was there are a lot of events when you edit files (temporary files), and even when you save, a file editor as vim can produce as much as 8 events for only one save action. So a few options were added, and i think it’s now very usable.

For clarity, pythonar will use colors if possible (using the termcolor module if installed, and using colorama for windows compatibility if installed). So you should easily distinguish the events that trigger reloading of your app.

So here is a little presentation of what i called PythonAR, for Python Auto Reloader (althought it can be used for any apps, not just python ones):

Assume you are editing a few files with vim, and want to restart your python app when you save any file, the base configuration for that would be :

reload python my_app.py

or

reload ./my_app.py

But you’ll run into a few issues, first, vim will create swapfiles, that will be changed after about everytime you touch your keyboard, let’s ignore those files.

reload -i *.sw* ./my_app.py

Now, when your app will start, it will often create .pyc or .pyo files, we can ignore them too :

reload -i *.sw* *.py. ./my_app.py

We’ll soon discover it’s not enought, thought, what happens when vim save a file is not a single operation, first vim test if it can write in the folder, creating a file named “4913″, and delete it, then it moves the files it want to save to a backup, create a new file, modify it, and delete the backup. That would trigger way to much restarts to be comfortable with. Filtering with -i, as we did before can only lower that to two restarts, because two events are happening on the real filename we want, so another option is needed.

the “-s” (or “–sleep”) option, will prevent any event to trigger a reload for the number of seconds. That will resolve this problem.

reload -i *.sw* *.py. -s 1 ./my_app.py

For graphical applications, most window managers will pass the focus to the newly created window each time it is restarted, this can be a serious distraction. The solution here is not perfect, and probably won’t work out of Linux (patches welcome, thought ^^). The “-f” (or “–focus”) option, will prevent the wm from changing the focused application for at least a few seconds, effectively preventing the new window to steal the focus. Unfortulatly, this depend on “xdotool“ an utility based on the xlib, so only useful on X. Any solution for window or OSx appreciated.

reload -i *.sw* *.py. -s 1 -f 3 ./my_app.py

Whatsmore, the -p (”–path”) allow to monitor a different folder than the current one. And the “-a” (or “–action”) is reserved for potential future actions, the default and currently only available value is “restart”.

Yeah, that’s quite a lot of options, i think the next useful addition would be some config parser use, to save options for a project in a simple ini file.

That’s it :) i hope it will be useful to people. The project is hosted on Github, as most of the stuff i do nowadays :).

quand mv et rename ne suffisent pas... python

written by tshirtman, on 11/28/11 5:49 PM.

Ayant pris une trentaine de screenshots, pour avoir un backup de mes slides pour une présentation (ce soir à la cantine, à paris, pour parler de kivy oui je préviens vachement à l’avance). j’étais un peu embété par le format de nommage par défaut de gnome, indiquant “Capture du … xx:xx:xx.png”.

J’ai réfléchis à une utilisation de rename, mais à part retirer la partie commune, je n’avançais pas beaucoup, car bien sur, je voulais en conserver l’ordre. Du coup, python à la rescousse!

# chopper toutes les images dans une variable
a = filter(lambda x: x.endswith('.png'), os.listdir('.'))
# s'assurer qu'elles sont bien dans l'ordre alphabétique (contenant l'heure, et donc, dans le bon ordre)
a.sort()
import shutil
for i, nom in enumerate(a):
    shutil.move(nom, str(i)+'.png')

 

Et voilà, les images ont un nom tout propre, et sont toujours dans l’ordre.

zine img_upload

written by tshirtman, on 10/24/11 9:20 AM.

Un autre petit plugin zine, tout simple, mais qui simplifie la vie, pour envoyer des images sur le blog pendant qu’on écrit un article.

Le fonctionnement est simple, un bouton javascript “click to upload image” est ajouté entre le formulaire de titre et de texte:

cliquer sur ce lien permet de choisir un fichier sur son disque, une fois validé, le lien vers l’image apparait en dessous du bouton javascript, y’a plus qu’a copier/coller :). Si une image du même nom existait, des caractères aléatoires sont ajoutés avant l’extension jusqu’à trouver un nom disponible, afin de ne pas écraser les images existantes.


Un panneau de configuration simple est disponible, pour configurer les deux paramètres essentiels:

  • Le dossier ou stocker les images, ce dossier dois exister, il n’y a pas de gestion d’erreurs pour l’instant, si c’est mal configuré, vous ne verrez juste pas de lien apparaitre :).
  • L’url pour accéder à ce dossier pour les clients, soit vous configurez votre hébergement pour avoir un dossier pour les fichiers statiques, soit vous vous débrouillez pour poser ce dossier dans la partie statique de zine, et vous donnez l’url correspondante.

Le code est dispo sur github: https://github.com/tshirtman/zine_upload_images.

image proxifier

written by tshirtman, on 9/11/11 4:42 PM.

Juste pour signaler un petit outil que j’ai réalisé pour dépanner une connaissance, celle ci ayant parfois “besoins” d’accéder as des images sur des domaines bloqués dans son entreprise, et qui donne l’occasion de montrer un peu de code flask. C’est ici, tout simple, ça ne marche que pour les contenus simples (image/vidéo), pas les pages complêtes, y’a sans doute moyen de faire mieux, mais c’était pour dépanner :).

zine_social

written by tshirtman, on 7/22/11 1:11 AM.

Juste pour annoncer un petit plugin zine que j’ai codé, permettant d’ajouter une barre de réseau sociaux en bas des posts de blogs, seuls google +1, le tweet this et le like de facebook sont géré pour l’instant, mais toute contribution est bienvenue, c’est vraiment juste ajouter le code du bouton avec la variable “url” à la bonne place, et zou…

Le code est dispo sur github, et vous pouvez normalement voir le résultat en dessous de ce post :)

Tags

#FIXME 3G absurd ad_sense alterway aléatoire android animation anonymity atheism bach backlog bash bitcoins blog blogging boldness book books boulot bricolage bépo C canvas captcha captures carte SD censure christianity chroot CI CLI cli cloudwatt code colors comfort zone command line community company life conferences contest copwatch copwatchnord-idf core-devs cours ct705 culture deb debian debug deformation dehors dessin dev distribute distribution docker débutant déploiement développement ebooks eeepad eeepc effect ego empty en escher event firefly flappy bird flask fosdem foss fr fun game garden gdb geek culture git github goals graphics grrr gödel hack hackathon hacked hacking health hooks i3 images IMAP inspirational install isync java jenkins jeu jeu video jinja2 jni keyboard keynav kivy kv lame learning lib libre life linux lol macosx magnet mail mailing-list mails maths mbsync meetings memory leak mesh meta mint mirroir MIT module motivational mouse museomix mutt nexus7 no-mouse notmuch nottoomuch offlineimap onycroit opencourseware osc packaging paris passphrase password patch pentacatyl people perte de données ping pip planning plugin positioning pr procrastination programmation progress project projet property proudhon proxy psf publisher/consumer pull-down pygame pyjnius pypi python pythonar qtile raid rapsberry pi reading recorder references release religion responsive review reviews réseau réseaux sociaux résurection salon screenshots script self service shows shutil shyness sizing solib sortie sousous!!! spam spritz stash status systeme système templating terminal texture texture coordinates Thomas Paine thread thème tiling time time management. tip tips tools transformer tutorial tv twitter typematrix typing ubuntu ubuntu-fr ultimate-smash-friends unity update upload images useless usf utils value VDM video vie/mort vim virtualenv visite widget windows wm wmii work workflow workflow. zine études