汉字标准格式

这个专为中文网页而设的 汉字标准格式（CSS）实在很棒！前些日子和 ChinaTeX 采访／闲聊时才说起，要找专针对中文内容的 Typography 样式，实在很难，没想到两个多月后就在 Twitter 看到了。

Recursively Fixing File Permissions

Ahhh, the joys of restoring/moving files from a FAT32 external hard disk to a *nix system. Everything is 777, grrrrr. The all-knowing Google kindly directed me to try the following:

$ find . -type f -print0 | xargs -0 chmod 644

to chmod all files to 644, and

$ find . -type d -print0 | xargs -0 chmod 755

to chmod all directories to 755. Ahhh, the joys of bash commands.

Mounting USB Devices in VirtualBox

Mounting USB devices on guest systems in VirtualBox seems to be a big headache for many people (yours truly included). But I think I’d finally figured it out.

It boils down to these two most crucial points for me:

1. Use the same usernames on your host and guest systems.
2. Create an empty USB filter for your guest system. This will automatically mount any devices to the guest system as soon as they are attached physically.

Use the same usernames on your host and guest systems.

• When installing your guest systems, create users with the same login name you’re already using on your host system.
• To make things easier, choose a username without space characters.

Create an empty USB filter for your guest system.

• With the guest system off, open the Settings of your guest system.
• Open the Ports → USB pane.
• Make sure you have enabled USB 2.0 (EHCI) controller.
• Click the icon with a blue circle on the right. This will add an empty device filter that matches anything.

Now try starting your guest system, log in with the same user name as the one currently active on your host system, and attach a USB device. Hopefully things will… work!

Facebook Query and Markup Language

Did you know FB has a Query Language and Markup Language of its own?

• Facebook Query Language (FQL)
• Facebook Markup Language (FBML)

Just in case anyone’s interested in these for their research. Also to KIV in case I find them useful in future.

Open-Source Licenses

This article gives quite a good overview of the various open-source licenses, written in an easy-to-understand way. Don't be fooled by FUD ever again.

LTNS MySQL

I've forgotten so much about MySQL administration ever since I embraced PostgreSQL. Baaaad me.

Batch Transcoding with VLC 1.0.x

After I upgraded my VLC player to v1.0.x, I realised my batch transcoding script stopped working. After much tinkering with the syntax and googling, I've got it working again. Here is the updated script:

#!/bin/bash
vcodec="mp4v"
acodec="mp4a"
bitrate="3072"
arate="192"
ext="mp4"
mux="mp4"
# Modify the path to VLC on your machine accordingly
vlc="/Applications/VLC.app/Contents/MacOS/VLC"
fmt="MPG"
dst="/Dest/Path/"
for a in *$fmt; do
  echo $a
  $vlc -I dummy -vvv "./$a" --sout="#transcode{vcodec=$vcodec,\
  vb=$bitrate,acodec=$acodec,ab=$arate}:std{mux=$mux,\
  dst=\"$dst$a.$ext\",access=file}" vlc://quit
  touch -m -r $a $dst$a.$ext
done

Begone, ._whatever!

So Finder in OSX has this quirky habit of creating a ._blah for every blah file when transferring file between operating systems, due to the AppleDouble system and resource forking. (No idea what those mean exactly, but you can always ask Google.)

David Olinsky left this nugget of a solution as a comment here:

export COPY_EXTENDED_ATTRIBUTES_DISABLE=true

UPDATED: as well as this attribute, introduced in Leopard as noted by nzight:

export COPYFILE_DISABLE=true

Thank goodness! Now stick those in your .bash_profile...

Oh and to remove those blasted files:

> find . -iname '._*' -exec rm -rf {} \;

Batch transcoding with VLC

I have a bunch of MPG videos from my videocam but neither iPhoto nor iMovie would recognise them. Now VLC has a transcoding wizard that'd do the conversion to MP4 nicely, but to do it interactively through the GUI is just too tedious. Using this tip from the VideoLAN Wiki, I used a script to perform the batch transcoding:

#!/bin/bash
vcodec="mp4v"
acodec="mp4a"
bitrate="3072"
arate="192"
ext="mp4"
mux="mp4"
vlc="/Applications/VLC.app/Contents/MacOS/clivlc"
fmt="MPG"
dst="/Dest/Path/"
for a in *$fmt; do
  $vlc -I dummy -vvv "./$a" --sout "#transcode{vcodec=$vcodec,\
  vb=$bitrate,acodec=$acodec,ab=$arate}:standard{mux=$mux,\
  dst=\"$dst$a.$ext\",access=file}" vlc://quit
  touch -m -r $a $dst$a.$ext
done

If you want to try this on Linux or Windows, you should use vlc instead of clivlc. I included the touch command so that the transcoded videos would still have the original capture date/time as their last modified date.

Formatting Large Hard Disks with FAT32

I badly needed a new external hard disk. My boss gave me one of 150GB — in NTFS.

Darn. I hardly use Windows these days, only GNU/Linux and Mac. And WinXP wouldn't let me format it to FAT32.

Darn darn darn.

Until I came across fat32format. All I did was to issue at the command prompt

fat32format e:

and the hard disk was formatted in, like, what? 2 seconds.

*Blink*.

Wheee!!!

EOL character conversions

I need to inspect/process large text files (mostly dictionary data) on a regular basis, and for a quick preview I'd use the trusty less. But one thing nearly always drives me crazy: end of line (EOL) characters.

So what's up with all the EOLs? Read all about them.

Luckily since I'm working on *nix systems anyway, there are nifty little utilities, nay, bash commands that'll do the EOL conversions in a blink.

For conversions between DOS files (CR-LF EOL) and *nix files, use dos2unix and unix2dos. (Or use sed if you're old school, as shown in this Linux FAQ HowTo.)

On the other hand, if you need to convert Mac files to/from *nix files, try out this trick with tr, or awk, or perl.

And finally, if you also need to convert between character encodings/sets, you might want to use the versatile recode utility instead. I use it all the time now since I need to process multilingual dictionaries very often.

iGoogle haz new lookz

iGoogle's just got a new look yesterday:



The tabs are now all lined up at the left of the screen, instead of across the top. This is quite alright if you have a wide screen, but for people like me who's got smaller resolutions, the screen real estate has really shrunk. Each widgets may now be "maximised" to fill up the whole content frame, like the Google Reader widget in the screenshot above.

My first impression: Whoa, it's like launching a new applications when you maximise these widgets, and something like a preview when you only expand them. Hmmmm... is Google going to make this into its own operating system? Y'know, do everything online, forget about installing OSes locally on your own machine and be tied to it?

UPDATE: iGoogle gives a tour of its new features.

Microsoft Office 2007 to support ODF?

If what's reported in this PC World article is true, I'd be so happy. I can then happily work away on OpenOffice and e-mail them to all the other Microsoft Office users without converting them and getting a bunch of tables and bulleted lists wrangled in a the process!

Wow. Hibernate worked!

On my Fujitsu laptop. Running on Debian. Wooow.

Debian yammered about starting the snapshot process (checkpointing, Hussein sez) and proceeded to write n number of pages. Once that's done, the laptop powered down, and I went out to dinner. When I powered on the computer again, what greeted me was the GRUB menu. And I was thiking "well so much for false hopes..."

Once I booted into Debian though, it started to yammer about resuming from n pages of image (checkpoint-restore, Hussein sez) and voila! It's back to exactly where I left off!

Both the checkpointing and restoring took longer than in Windows, though.


ps: Apparently Hibernate is not expected to work usually:

<obiwan> hey guys
<obiwan> 'Hibernate' actually worked for my colleague.
<ahu> .. once
<ijuz> probably some new drug
<obiwan> No, it really worked

Hahahahahahaaaahahaha! :D

English-Malay Translation Google Gadget

So I was bored (or having a writer/researcher's block, depressed to tears, too busy P-ing, no research inspirations, whatever). So I decided to try writing a Google Gadget for UTMK's EBMT solution:



Basically, it's a "lite" version of our research group's example-based machine translation solution (read: a Universal Translator in the making). For the moment it supports English-Malay (both directions) translation, and we still have whole tons of ideas for improving its speed, accuracy, coverage etc.

I tried submitting the Google Gadget to the public searchable Directory, but apparently there's an approval process, so it doesn't show up yet. Meanwhile if you'd like to add it to your personalised Google homepage, you'll have to add the gadget by its URL: http://liantze.googlepages.com/utmk-ebmt.xml

Stay tuned!

Bash commands and scripts

I love Bash shell commands and scripts. Really, I do. Where would I be without all them one-liners to *bang* *zip* process those huge dictionary text files in less than 5 seconds?

Except that sometimes I need 30 minutes to look for an obscure option and how to use it exactly to get the result I want.

For instance, sort is cool. It sorts lines so fast no spreadsheet application is worth a second, nay, even a first thought. Heck, it even ferrets out unique lines calmly!

Then I have this file, which is all fields of numbers, e.g.

1   1    0    0    0
1   2    2    0    2
...

And I needed it to be sorted by the first field, then the second. Running plain sort on it gave me

1     1     0     0    0
1    10     0    10   10
1    11     ...
...
1  1921   ...
1     2      ...

OK, so it's not seeing the numbers as numbers, it's seeing them as ASCII strings. A man page look-up later, I tried sort -n. Then sort -k, then with both options. To no avail. Time to Google.

30 minutes of surfing later, I finally found this nugget of an example that did the trick:

sort -k 1n,1 -k 2n,2

See, you have to state explicitly (via -k) that the first key starts and ends at field 1, and to treat it as a number, and repeat it with field 2 (or any other fields you'd like to use as the sort key) as well.

I love Bash commands.

Debian 扫除

我本来是看着 df -lh 的列表，暗自神伤……怎么那么快硬碟就不够用了呢？我又不是装了多少软件包，不就是 LaTeX, 两个资料库平台，Apache, Java, Prolog, Perl, PHP……嘿嘿 ;-)

然后！然后我发现了一个沧海遗珠！那就是——

apt-get autoclean

它可以把陈旧的 .deb 文件清洁溜溜！了无痕迹！（洗衣粉吗？）若要更狠一点，把全部 .deb 都删除，就用

apt-get clean

包您的硬碟马上又有一大片空间！

Java 6 on Linux!

哇～！哇～！我不是眼花吧？ Sun Java 6 的 Debian 包出了啊啊啊啊～！而且还顺利安装了呢～！而且之后的执行速度真的很快！大感谢，大感谢！

Jawbone! WordNet!

终于找到 WordNet 2.1 的 Java API 了！看看 Jawbone 吧。

又，Ted Pedersen 等人的 WordNet::Similarity 也看来蛮有用的。但是个 Perl 包……我看了就头疼。暂时我会先用 HTMLParser 来抽出那些数据，留待他日／他人再研究把它弄成 Web service 好了。

我爱 PostgreSQL

我真是爱死 PostgreSQL 了。可不是要又来一场 MySQL vs PostgreSQL 激烈辩论哦。只是因为我受够了每每翻阅 MySQL documentation 时，正开心着终于找到某个 function 或什么时，赫然看到这么一行字："This function will be available in a future release of MySQL." 真是气死老娘也。

不过哪，PostgreSQL 也得小心服侍就是了。就比如在 PostgreSQL 8.x 里的 constraint_exclusion 变数和 table inheritance partitioning 若一起使用，可以得到很好的速度，但 constraint_exclusion 在 PostgreSQL 7.x 可是没有的哦。还有还有，PostgreSQL 8.x 可以用 $$ 或 $_$ 之类的 string delimiter，PostgreSQL 7.x　可就非得用 '' 不可。害得我在那架还是 Debian Sarge Stable 的伺服器上要一个一个 function 的去改正。

最后，还发现了一件事：若一个 用 SQL 定义的 PostgreSQL function 里有用到 where condition 而这个 where condition 又用到 function argument 的话，执行的时候可有您受的：因为 Query Planner 事先 （compiling function 时）根本不知道那是什么值，因此它完全不会打算用任何 Index！！！遇到这种情况，就得用 Pl/PgSQL 来定义那个 function。代价是每回执行时都要 plan 过一次 query，但我甘愿！因为我的 function 执行时间就此从七秒降到零点七秒。呵。呵。呵。呵。呵。