14th May 2018Argument parsing in python
Almost 10 years ago, I thought about implementing a new solution for arguments parsing in python.
Basically, existing solutions (now and then) instruct a parser on the expected options and flags, and the parser handles the arguments, catching any errors and producing a beautiful help summary automatically. The major issue is the flexibility of the parser to define the correct arguments syntax: for example, whether it is possible to specify incompatible options, flags, etc.: two options --verbose and --quiet can be incompatible, or action scan could require a mandatory --source option. Additionally, if the parser is flexible enough to provide this functionality, it can be quite difficult to program, or to make changes at a later stage.
My idea was to define class methods that could handle each possible combination of arguments, and the parser would extract that information without further effort from the programmer. That is: express what each operation requires, and expect that the parser would handle all the required logic:
class Example(OptionMatcher): @optset def handle_common_flag(self, mail_option): ... @optmatcher def handle_compression(self, file, compress_flag=False): ... @optmatcher(flags='verbose', options='mode') def handle(self, file, verbose=False, mode='simple', where=None): ...
It does not only simplifies arguments handling, and expresses clearly the purpose, but adding / removing options or flags or operations is in fact very easy
I implemented this solution shortly after, and then started adding functionality as required. Looking to the history of the project, I invested around 10 months on this projects, although I do not remember anymore the associated effort -the optmatch.py file has just 800 lines of code, plus comments, plus tests and documentation-.
And then I used it in some of my projects, a few people contacted me about it, and it was included in a some other projects, but it was definitely not a success. In fact, when I needed to do arguments parsing on scripts at work, I would normally default to a standard solution argparse
A few weeks ago I was once again in the dilemma of creating a minor personal application, and I started using argparse. Soon I got into the familiar territory of having incompatible options, flags for only specific actions, and to express different actions under the same script. This meant adding code after the parsing to handle all these issues. And, after a while, adding a new option required quite a lot of effort just to handle all properly, so I remembered my own unloved library, and decided to give it a try, again.
And that was it, like falling for some old love :-)
So I have spent a few hours updating the library -it was only supporting camel case notation on parameters, and now it supports the more standard underscores-, and, more importantly, uploading it to the standard Python Package Index (PyPI), so it can now be easily installed as:
pip install optmatch. Even better, support for python 2 and 3 is now included in a single file.
11th December 2017Bollocks UI
I read this interesting article on the (wrong) effect of using flat UI, where users are found to require longer times to grasp the meaning of the UI elements in web pages, delaying therefore their actions.
Bollocks, but not so much as trying the latest Nautilus interface, the default file navigator in Ubuntu 17.04. Creating a new folder requires double clicking on the folder contents and selecting the firt option (New folder). But note that selecting the parent folder and right clicking shows a menu where the option to create this new folder does not appear. Furthermore, Nautilus has an application menu, but the option to create a menu is all but missing.
When you right click on the folder content, the first option is indeed 'New Folder', and it displays very handily its shortcut: Shift+Ctrl+N. BUT: what happens if the folder contains too many files/sub folders? There is no way to right click on empty content, and there is no way to create a new folder except by knowing the shortcut.
I guess that one or more developers went just too far on their quest to simplify the interface.
5th December 2017OS Agnostic
Yep, after 6 months in Ubuntu land, the Wayland switch -or the associated bugs- dropped me back into the arms of Hackintoshing
Not happy about it, but Wayland is at the moment no go in my configuration; and MacOs installation was really simple, just a couple of hicups. The whole installation process is described here
Switching operative systes is now an almost painless process. I do not rely on the cloud to store my files -just a few ones, which in fact host most of my configuration information. As a result, installing a new operative system normally implies:
- The OS installation itself -30 minutes for Ubuntu, a few hours for Hackintosh
- OS Configuration, a 15 minutes process once I have it correcly documented
- Programs installation, a 60 minutes process, using software centers and command line
- Copying my common folders, either from the old drive or from a backup, which takes up to 3 hours-.
5th December 2017Ubuntu 17.10
Six months ago, after repeated issues with Hackintosh, I moved to Ubuntu 17.04, and all was well.
Then it came Ubuntu 17.10, with the move from Unity, and the introduction to Wayland, and man, was that a big change. It was supposed to be, like every last Ubuntu releases, a minor upgrade, almost more of an update than a real move.
But. That workspaces do not work anymore as they used to do with Unity, that shortkeys seem to only work randomly, that Steam seems much slower and it seems to exhibit a gap between the cursor pointer and the exact pointed location are for me just inconveniences; but that whenever the monitor goes to sleep, Wayland performs a swift Harakiri, with all X programs getting killed, is a major showstopper. I cannot justify all the lost time.
What is special on my setup to produce such an outlier crash? No idea, I use a embedded Intel HD 4000 solution with DisplayPort connection; I have tried disabling DCI on the monitor, without success. For me, Ubuntu 17.10 is, on this machine, no go.
I have filled a bug for this problem , but no solution or activity so far...
I could stay with 17.04, but that means a very short security period with upgrades. I could try installing 16.04, a LTS solution, or, as I have done, let the upgrade Ubuntu 17.10 in place, hoping for updates to solve my issue, and have a new Hackintosh try, on a separate drive. Let's see how the fight Ubuntu Wayland vs Hackinstosh works for me in the close future...
Note: on 28th November, I have switched back to Ubuntu, upgraded, dist-upgraded and rebooted. But Wayland persists on its suicidal ways....
1st May 2017DisplayPort on Ubuntu
There is an ongoing issue with using monitors in Ubuntu connected via DisplayPort: monitor shows black screen and "no input signal" after turning the monitor off and on manually
Turning off a DisplayPort-connected monitor is treated the same as disconnecting it, and somehow X11 does not recover from this. I have seen this error related to Nvidia and Radeon cards, but in my case I have a Intel HD4000, and the error is exactly the same.
And it happens with just all kind of monitors, including my Dell U3011. A proposed solution is to disable DDC/CI on the monitor itself, but this didn't solve anything for me.
A solution I have found is to press Ctrl+Alt+F6 (or +F5, etc), to open a TTY console, and then pressing Ctrl+Alt+F7 to get back to X11. But it works sporadically, sometimes having to press these keys several times, or creating new TTY consoles, like pressing Ctrl+Alt+F2 to create a TTY2 if the 6 had been already created before.
Other provided solution is to ssh from another machine and run:
env DISPLAY=:0 xset dpms force off env DISPLAY=:0 xset dpms force onIn this case, the best option is to create a shell script /usr/local/bin/display-port-wake-up.sh with this content, open Settings, Keyboard, Shortcuts, and create a custom shortcut (in my case, Ctrl + Alt + W). This solution works always, but there is a catch: the shortcut is only available when the user is logged in, so if the system is asking the user password to unlock the screen. the shortcut will not work. In this case, it is needed to enter blindly the password, press Enter, and then press the shortcut.
30th April 2017Moving Ubuntu to separate disk
After a few weeks with my new Ubuntu installation, I was able to do all my usual tasks without missing OsX. So far, I have found only two issues: display port monitor not awaking some times, and some crashes in Steam.
But the main issue has appeared when trying to it setup for Android development... and running out of space. So, the original hackintosh disk, which was in standby, had to go, and the idea was to clone my existing installation to the other disk.
So I reformatted the hackintosh drive, just to find that I had removed the EFI partition that booted the Ubuntu system... My solution was to launch the Ubuntu installer, and install Ubuntu again on the new drive (the previous Hackintosh drive), taking care of having one EFI partition, plus a big ext4 one. Once I had Ubuntu installed, I launched again the installation system, mounted the old partition under /media/old, and the new one under /media/new, and then copied all the important files:
sudo cp -R --preserve=all bin/ etc/ home/ lib/ lib64/ opt/ root/ run/ sbin/ usr/ var/ /media/new/
Then edit the file /etc/fstab to change the UUID of the disks, and presto!
13th April 2017Setting up Ubuntu
Setting up a Ubuntu computer seems to be my fate of late. And each time I do it, I need to do the same Google searches: how to configure workspaces, how to define my shortcuts, how to make Guake start at runtime, etc
So I have collected all the steps I take to configure a Ubuntu machine, from a generic point of view, not describing all the applications I finally use, but definitely including all details to configure Ubuntu as I like it. It seems weird to show these steps for a Ubuntu 16.10 installation on the same day that Ubuntu 17.04 is published, but I will definitely comment on any changes.
12th April 2017Ciao, Hackintosh
I have been setting up my computers as Hackintoshes since 2009; currently I have a Mac laptop, with the latest Sierra installation, and a Dell laptop, running Ubuntu 16.10, plus two desktop computers, one running Windows Vista and Snow Leopard (yep, both run still perfectly fine), and the other with MacOs Sierra. At work I use exclusively Linux, and I had been wondering for a long time on my reasons to keep my Hackintoshes at home.
As of last weekend, this question has been answered, fare well, Hackintosh. I will keep the old desktop running Snow Leopard, and my Apple laptop running MacOs, but I definitely see no point on not using Ubuntu as my first OS choice. My choice is rather simple to do, as none of the programs I use lack a Linux version, with the exception of Evernote, which I can still handle via its Web interface.
10th January 2017Batch editing Google contacts
The new Google contacts application is very nice. It looks great, and it offers good functionality, like merging of duplicates. Editing contacts works perfectly, but it can only be done one contact at a time. I was migrating contacts from a non smartphone, and the migration had converted them into an ugly 'Family name; surname' format, such as 'Trumpy; Donald'. And as slick as the user interface is, editing by hand over 300 contacts was a boring perspective.
Automation to rescue: export contacts, process them with a python script, import them again. Do, in Google Contacts, press 'More', then export, then read the popup warning and head to the old Google Contacts application, as the new one seems to be unable to export the contacts. Press again the More button, again on export, and choose Google CSV format.
The script to convert the contacts is as simple as:
import csv with open('google.csv', newline='', encoding='utf16') as f: contacts=list(csv.reader(f)) for contact in contacts: colon = contact.find(';') if colon > 0: name = "%s %s" % (contact[colon+1:], contact[0:colon]) contact = contact = name with open('google-out.csv', 'w', newline='') as f: csv.writer(f).writerows(contacts)
This reads the exported google.csv file, and creates a new google-out.csv file; in the old Google Contacts application, remove now all contacts, and then initiate the Import process, passing the created file. Easy as pie.
The previous script shows a very basic transformation; the important aspects are: (1) it is needed Python 3 to run this script, as its CSV reader handles properly the unicode format. (2) The input format seems to be UTF-16 (it was definitely on my OsX machine). (3) However, it was reimported as UTF-8 without issues.