6th April 2013Autocounters in Django ORM

django iconFighting today in Django ORM, how to use a typical autocounter. Let's say we have a database table with two columns called name and value, being name the primary key. The idea is to be able to increment the value associated to a key, that will be used for some external purpose(s). As the Django documentation mentions, this implies a solution such as:

product = Product.objects.get(name='Venezuelan Beaver Cheese')
product.number_sold += 1
product.save()

And the same documentation shows the race condition problem: two thread or processes could perform the same operation, resulting in the counter being increased only once. The Django ORM solution makes use of F() expressions:

from django.db.models import F
product = Product.objects.get(name='Venezuelan Beaver Cheese')
product.number_sold = F('number_sold') + 1
product.save()

With this code, the counter is effectively robust and better performant. The problem is, how to get hold on the incremented value? It is needed to reload the object to access the actual value:

sold = Products.objects.get(pk=product.pk).number_sold

Which brings back the original race condition in a different way: now, two simultaneous processes would effectively increase twice the counter, but both could get the same counter value!

This is not a problem of Django, but of the underlying database, and F() expressions can still be helpful if the counter's value is not required. If this is not the case, trying to overcome the counter problem using exclusively the Django ORM approach is an invitation to lose a few hours testing all possible approaches :-(

22nd March 2013More sqlite3 tables alterations

database iconDuring a current migration of a project to Django, I wanted to make explicit the usage of a primary key on a table that lacked it (using the default rowid column instead).

My previous release of alter_table did not support renaming the rowid column, so I have released a new version (0.20) with this support. Adding some more functionality, the utility also supports now reordering columns in tables.

17th March 2013Altering sqlite3 tables

database iconsqlite3 does not support some basic SQL operations, more specially dropping or renaming columns.

alter_table is a python utility that supports specifically these two operations. It modifies the tables and any related indexes; however, if there are views or triggers depending on the altered tables, the user must recreate them.

14th March 2013PI day!

brain puzzles iconStumbled by mistake on a wikihow page about How to Celebrate Pi Day. No words to describe it.

Fortunately, I also stumbled on this page on Epigrams on Programming, by Alan J. Perlis. Some jewels, in the order they appear:

And, just for fun:

12th March 2013TableFilter v5.0.0

table filter iconI just released a new version for table filter, a library to enable Excel-like filtering on Swing tables.

Although the functionality changes are rather minor -basically affecting the sorting order of choices-, it had quite a large impact on the implementation. However, the main reason for a major version change (from 4.5.0 to 5.0.0) is because the support for Java 5 is finally dropped. Java 5 always lacked the required functionality to support table filtering, so a lot of code had to be maintained to provide that support. And that work is now difficult to justify: mind you, Java 5 entered its end-of-life on April 8, 2008 and is no longer supported since November 3, 2009.

So, good riddance, to the Java 5 support!

3rd March 2013Playing with OpenWRT

router iconI bought a cheap router, a TPLink 841N, for the explicit sake of installing OpenWRT and playing with it (which originated in the unfulfilled wish to setup a SSH tunnel on my existing Netgear router with the existing official firmware).

My main objective is very simple: I have hosts on several locations, where I run OpenVPN, and I would like to automatically being able to appear as having an IP address on those hosts. That is, by connecting to specific ports on my router, all my network traffic will appear as originating in one of those hosts.

So far, I have succeeded at setting up correctly the wired ports, and I hope to have ready the wireless connections sometime soon.

7th December 2012Mountain Lion clean install on Snow Leopard Macbook Pro

macbook pro iconI purchased Mountain Lion a for a new Hackintosh, and decided finally to upgrade my reliable Snow Leopard in a early-2011 Macbook Pro. I thought it would be better to do a fresh install: I am rather active keeping backups of the important stuff-, so this would be a great opportunity to get rid of so much software I had been installing during the last 20 months.

But Apple had preferred to not offer such option to its customers -for the sake of simplicity, as I guess that is what the majority of users do-.

One option to do this clean install is (see link) to make a bootable USB drive, copy there the required installation files and boot the Mac with it. But I was rather curious on the usage of the recovery partition -where a fresh install can be done as well-. So I created a exact copy of my hard disk -disk utility, restore to separate partition (on a external USB disk)-, to avoid any problems, and booted into the recovery partition.

Now, this was one of my points of interest. My Macbook Pro lacked such recovery partition, so the process involved a download of the image, from where it boots -all transparent to the user, just slower than in the presence of the physical partition-. Then I was greeted, among other, with the disk utility and the option to install OsX. I erased my hard disk, and proceeded to the OsX install.

First doubt: the system displays clearly that it will install Lion, in so far as my computer is eligible to do so. I didn't want to install Lion, and I was in fact not eligible for a free upgrade, and I hadn't purchased a Lion license. All the same I continued with the process. A Mountain Lion install on top of a fresh Lion install would not be that bad, and perhaps the process would connect to the App Store and see that I had a Mountain Lion license, so perhaps the option to install Mountain Lion would be offered at some point in the installation process.

I was almost right: I am prompted to introduce my App Store credentials and..... the installation is aborted, complaining that I have no Lion license. That was indeed a shame: the whole process was almost perfect, but the BIOS had no was to handle future OsX updates.

In that moment, my Macbook Pro was half way its target: was totally clean :-)

Fortunately, I could complete the process by booting into the external USB disk and launching there the Mountain Lion installation -to install it on the laptop's SSD, of course-.

Summary: if you are going to do a fresh install, you should normally have a full backup first, and the disk utility erase/restore process is perfect for that. At that point, the best option is to boot into that external disk and launch there the disk utility to erase the laptop's hard disk, and then the Mountain Lion installation package.

17th November 2012New mobo: GA-B75M-D3P

mobo iconI just bought a new motherboard, a Gigabyte GA-B75M-D3P: its target is linux and / or hackintosh, and I have no plans at all to install here Windows (8 or 7).

I do not intend any overclocking or so, and my only requirements was to have DisplayPort support to drive a 30" monitor, and SPDIF output, and this mobo got it all, for 75 euros. Jumping up the specifications to a H77 or Z77 mobo would not give me any additional value.

This mobo is targeted to business users, and it is definitely unassuming. It does not even have any led to show that the PSU is connected. But I installed directly 32 Gb of memory and it recognized them without any problems.

The only issue I have run so far into is on the BIOS update. All my previous mobos, from ASUS, support flashing the BIOS from inside the BIOS setup. Some Gigabyte mobos have similar functionality, called Q-Flash, but not this specific one. More to the shame, the manual points to this non existent utility to update the BIOS.

The download itself is an .EXE file. Fortunately, it is just a self extracting zip file, so it is possible to extract the contents using, in Linux: 

7z x mb_bios_ga-b75m-d3p_f6.exe

The contents include an autoexec.bat file, the flash utility (efiflash.exe) and the BIOS itself. Now, that Efiflash.exe can only be executed in Windows.... or, DOS. So, the solution is not so easy as to have flash support in the BIOS, and makes for additional 25 or 30 minutes of work, but at least is doable:

  • PS: shame on Gigabyte for such a loose implementation, specially when the mobo seems indeed solid.

    • 17th November 2012TableFilter v4.5.0

    table filter icon New version, 4.5.0, for the library.

    This release fundamentally introduces new functionality to handle properly the rendering of cells with html content. As minor update, the filter' constructor allows defining directly advanced capabilities to populate the auto choices directly, as this was the favourite way to use the library -even if I would hardly recommend it-.

    Available already on the central maven repositories.

    7th September 2012SSH keys

    SSHI have been using SSH for ages, mostly like a secure replacement to telnet, without really entering to understand all the stuff behind, and worse, without getting more benefits out of all the advantages that SSH and SSH tunneling provides. Still a long way, but enough now as to write a tutorial on usage and setup of SSH keys, and of serving subversion on SSH.

    2nd September 2012PSU rampage

    PSUTwo weeks ago, one of my hackintoshes started misbehaving. It would go to sleep, but the wakeup process never completed, even the monitor would stay without line signal. I tried booting into Linux, but the same problem reappeared, so it was a problem in the hardware. Didn't have much time to elucidate on the possible reasons, as briefly after, the computer refused to start again. Better said, it would start, auto power off and try to start again, without beeps or any screen information.

    These website was my best reference, specially How To Troubleshoot a Computer That Turns On and Then Immediately Turns Off. The problem seemed to be on the power supply, so I quickly opened my other hackintosh, and plugged its power supply into the dead computer: immediate success, time to change the power supply.

    The guilty PSU was a Be Quiet 300 Watts, about 3 years old. I failed to check the manufacturer warranty, which (at least now) covers 5 years, and without considerations on buying for second time a PSU of the same brand I purchased a Be Quiet 350 Watts. Plugged in -of course having to disassembly almost completely the computer-, and working again without (further) problems.

    Matter is, my other hackintosh PSU's broke down just a couple of weeks later!

    This time there were no symptoms that the guilty component would be the PSU, and I could not plug the other PSU this time -I needed a P4+4 connector, missing on the Be Quiet 350 W-. Fortunately I had a voltmeter and with the help of this reference page I was able to quickly verify that my +12 VDC line was supplying 12.92 volts, outside the ATX safety margin.

    Again, time to shopping; my previous PSU was a Cooler Master Silent Pro M500 (500W), with single rail. This single rail was rather problematic -To swap hard disks using the hard disk rack I had to put first the computer on standby-, so I did first some googling to find the best alternative. The best help I found was a calculator to find the required power for my system, but at the end I had to buy what my local dealer had in stock: a Be Quiet PCGH Edition -500W, 80 Plus Gold-.

    So, 100 euros poorer I could only wonder why both PSUs would die in such a short time. Initially I thought it would be due to letting the computers on suspend mode (aka hibernating for Windows), but this should not shorten specially the lifespan of the PSUs -in addition, one PSU was about 4 years old, the other 3 years old-. Other cause would be a voltage spike, but I do use surge protectors -Belkin- and seem to be working properly, so I haven't yet ruled out having had just very bad luck.

    28th August 2012Playing with Python on the GIMP

    gimp icon For a website I am developing, I wanted to use an image map where the user could specify a given position. The image would be simple, just small circles on predefined positions, and the user would need to select just one such circle. This seemed a very suitable scenario to use PythonFu, the scripting language in GIMP that allows to create images programmatically (or alter them, or define plugins).

    The lack of documentation (and experience) made the experiment longer than required, but it was a great way to get into PythonFu. Here is the script that I created, together with the associated javascript and my comments on the whole experience.

    23rd August 2012TableFilter v4.4.0

    table filter icon New version, 4.4.0, for the library.

    It is a minor update, although some interfaces are updated, hereby the version bump. The release focuses on the capability to keep the status of the filter header across executions: the editor interfaces expose now functionality to extract and restore the filter's history, and the classes supporting custom choices implement now the Serializable interface.

    Available already on the central maven repositories.

    12th July 2012Sublime Text 2 and .erb formatting

    Sublime Text 2 For the last weeks, I have been using the editor Sublime Text 2, and I find it rather good, with the huge improvement over TextMate of being cross platform. Obviously Emacs or Vi have already this advantage, but after many years using them and being at best only semi-proficient, I am really open to use other editors.

    The plugin architecture is great, and after installing the required package control plugin, it was time to start enhancing the editor -check this link for some tips on using this editor-.

    Sublime copes obviously with .html files, but it lacks an auto formatter. I installed Sublime-HTMLPrettify, that formats properly .html files, but alas, the rather requivalent .html.erb files are left untouched. The solution is just to edit the file run.js, located in Packages/Sublime-HTMLPrettify/scripts and convert the lines:

    if (source.match(".html" + "$") == ".html") {
	log(style_html(data, option));
}
else if (source.match(".css" + "$") == ".css") {
	log(cssbeautify(data, option));
}

    into:

    if (source.match(".html" + "$") == ".html") {
	log(style_html(data, option));
}
else if (source.match(".html.erb" + "$") == ".html.erb") {
	log(style_html(data, option));
}
else if (source.match(".css" + "$") == ".css") {
	log(cssbeautify(data, option));
}

    Obviously it just formats the .html.erb as pure html file, but so far seems fine to me.

    12th July 2012Recovering SVN revisions

    Rsync For my current development, I am using SVN repositories hosted at slik subversion, but I still keep some older respositories on my own server -precisely the server that crashed badly last weekend-; with backups rather up to date, the recovery was rather fast, and I was only faced with some checkouts that were now desynchronized: that is, on my development machine I had a revision that was newer than expected, and I could just not commit the changes.

    With rsync, the solution was very easy: checkout the most updated version from the server, and synchronized the version on my development machine, ensuring that the .svn files where untouched:

    svn checkout REPOSITORY website.svn
cp -R website website.backup                      #just in case
find website -name ".svn" -exec rm -Rf \{\} \;    #error messages can be NORMALLY dismissed
rsync -vr website/ website.svn/

    19th June 2012Installing Nginx on OsX

    NGinx Installing Nginx in Osx should be as easy as downloading the available source distribution, and compiling it; the only issue in OsX is that Nginx requires PCRE (Perl compatible regular expressions), and this library is only partially available in Snow Leopard.

    So, to install Nginx, it is needed to download PCRE, in addition to nginx, uncompress both files into any temporal folders, and type:

    cd /tmp/nginx-1.3.1
./configure --with-http_ssl_module --with-pcre=../pcre-8.20 --with-pcre-jit --with-http_ssl_module
make
sudo make install

    This will install it in /usr/local/nginx; To install it in another location, add the option --prefix=target to the second line above (./configure ...)

    9th February 2012ICS on Galaxy S

    Samsung Galaxy S i9000 icon On my Galaxy S I9000 I had installed DarkyRom quite a long time ago, and, overall, I was pretty satisfied with the speed and usability. I had replaced the launcher with the excellent ADWLauncher EX, and using the SwiftKey X keyboard, plus, of course, customizations on the themes, icons, etc. However, I was experiencing lately too many application crashes, so I decided to get some more stable ROM.

    After considering the MIUI and cyanogenmod ROMs, I backed up the system and installed the stable mod of Cyanogenmod (CM7). But even applying the extra download for Google apps, I could not get GMail installed -or any of the Google apps-. After a reinstallation -fast and painless, but also fruitless-, I planned to try the MIUI ROM... but then decided on the experimental Cyanogenmod CM9 based on ICS.

    The XDA forum, plus this page provided the complete information to download and install it. So I have been running it now for a few days (build 15) and it works fantastic.

    As most reviews point out, ICS looks and behaves much greater than Gingerbread (I have not seen any Honeycomb device); but overall, it looks initially like a subtle refinement on the GUI. However, these refinements just kept summing up, until I realized that I was greatly impressed with it. And much of this effect is due to the Roboto font, indeed.

    Then, the added bonus of having (finally) a Chrome version on Android is just priceless. And even better, knowing that my 2 years old phone is handling it properly; I cannot recommend it enough, to install and try this version on the Galaxy S. I bet it will stay, only to be eventually replaced by a later beta or the final CM9 version.