Seraching the text data on clipboard

Sometimes I need to use grep command on the certain text that I am looking at in an editor. Most of them do not have good grepping facility and even if they have there is not way to feed the results to some other process for further processing.
I ended up using a tool from nirsoft called nircmd to dump the clipboard contents in a text file and then use grep from cygwin. Here is a small 3 line script that I put in my bin folder along with the nircmd tool.

$ cat /cygdrive/c/cygwin/bin/clipGrep.sh
#!/bin/bash
MY_TMP_FILE=/cygdrive/c/tmp/UIUHUTGGUYGUhhjshjhasj_
rm $MY_TMP_FILE
nircmd.exe clipboard addfile `cygpath -w $MY_TMP_FILE`
cat $MY_TMP_FILE | grep $*

Now I can take any text on clip board and just execute this to filter what I want want. For example I selected all the text on Yahoo finance and took on clipboard (Ctrl-C) then searched for word "sell"

$ clipGrep.sh  -i sell
    Sell in March and Go Away?Matt Nesto

May not be a bad advice!

Running debian-mips in Qemu (Two NICs)

Note: The host system is Ubuntu and guest is Debian (mips) in the example below. In this example, eth0 of the host and guest are through the one NIC while eth1 of host and guest are through other NIC. There is a DHCP server running somewhere that gives IP to eth0 of host as well as eth0 of the guest. I run a DHCP server on eth1 of the host to give IP to eth1 of the guest. To install Debian in QEMU, use this (creating the qcow file is being left as an exercise to the reader, if you can't do even that, don't bother reading further): wget http://ftp.debian.org/debian/dists/Debian5.0.6/main/installer-mips/current/images/malta/netboot/initrd.gz wget http://ftp.debian.org/debian/dists/Debian5.0.6/main/installer-mips/current/images/malta/netboot/vmlinux-2.6.26-2-4kc-malta qemu-system-mips -M malta -kernel vmlinux-2.6.26-2-4kc-malta -initrd initrd.gz -hda Linux.qcow -append root=/dev/ram c                                            onsole=ttyS0 -nographic I tried with several other machine types beside Malta but they did not work. QEMU just hung there. The  interfaces file on host (/etc/network/interfaces) needs to be configured to use two bridged interfaces # This file describes the network interfaces available on your system # and how to activate them. For more information, see interfaces(5). # The loopback network interface auto lo  eth1 br1 br0 eth0 iface lo inet loopback #iface eth0 inet dhcp iface br0 inet dhcp         bridge_ports eth0         bridge_maxwait 0 iface br1 inet static         bridge_ports eth1         bridge_maxwait 0         address 192.168.41.4         netmask 255.255.255.0         broadcast 192.168.41.255         network 192.168.41.0 iface eth1 inet static         address 0.0.0.0 iface eth0 inet static         address 0.0.0.0 QEMU if up file (/etc/qemu-ifup) looks like this: #!/bin/sh echo "Executing /etc/qemu-ifup" Args= $* echo "Bringing up $1 for bridged mode..." sudo /sbin/ifconfig $1 0.0.0.0 promisc up if [ $1 = "tap0" ]; then echo "Adding $1 to br0..." sudo /usr/sbin/brctl addif br0 $1 fi if [ $1 = "tap1" ]; then echo "Adding $1 to br1..." sudo /usr/sbin/brctl addif br1 $1 fi sleep 2 Note: Make sure that brctl shows this information: $brctl show bridge name bridge id STP enabled interfaces br0 8000.000c29b09c78 no eth0 tap0 br1 8000.000c29b09c82 no eth1 tap1 if not then use this command to add tap interfaces: brctl addif br0 tap0 brctl addif br1 tap1 To run use this: qemu-system-mips -m 256 -M malta -kernel vmlinux-2.6.26-2-4kc-malta -hda Linux.qcow -append "root=/dev/hda1 console=ttyS0" -nographic -net nic,macaddr=00:16:3e:00:00:01,vlan=0 -net tap,vlan=0 -net nic,macaddr=00:16:3e:00:00:02,vlan=1 -net tap,vlan=1 Tip: If you encounter dns related issues, its easier to use opendns as the name server. You /etc/resolve.conf can look like this  to use opendns as the name server: domain localdomain search localdomain nameserver 208.67.222.222

Exporting bugs from Gforge.

At times you need to export data from a list of bugs for reporting purposes. A not-suggested way of One way of doing that is to export data from Gforge data tables using sql.
A quick way is to perform screen scrapping. I am giving to give example of this approach.

Here are the steps.

1. Log into Gforge account.
2. Get the details for a bug.
3. Fix the HTML to make a xhtml.
4. Use xslt transform to transform the xhtml into your desired format.


1) Login to GForge account.
You need to use wget to login. The wget invokation should be like specified below:
$ wget --keep-session-cookies --no-check-certificate --save-cookie=saved.cookie --post-data="return_to=%2fmy%2f&form_loginname=YourLoginName&form_pw=YourPassword2&login=Login" "https:/www.yourgforgeserver.com/account/login.php?return_to=%2Fmy%2F"
if you need to supply a http digest password then you should add this stanza in the beginning.
--auth-no-challenge --http-user=YourDigestUserName --http-password=YourDigestPassword
After this command the cookies will be saved in saved.cookie file.

2. Get the details for a bug.
$ export BUG_ID=7777
$ wget --keep-session-cookies --no-check-certificate -O $BUG_ID.details.html --load-cookie=saved.cookie "https://www.yourgforgeserver.com/tracker/?func=detail&aid=$BUG_ID&group_id=15&atid=165"
Note: The group_id and atid will be different for your project. Please specify correct values to get the details page.

3. Fix the HTML to make a xhtml.

The html generated by gforge is not wellformed so you need to clean it up. Use tagsoup to achieve this. Download the jar file. I used tagsoup version 1.2 from this (http://home.ccil.org/~cowan/XML/tagsoup/tagsoup-1.2.jar)location .
$wget http://home.ccil.org/~cowan/XML/tagsoup/tagsoup-1.2.jar
--2009-10-10 23:19:40-- http://home.ccil.org/~cowan/XML/tagsoup/tagsoup-1.2.jar
Resolving home.ccil.org... 192.190.237.100
Connecting to home.ccil.org|192.190.237.100|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 90023 (88K) [application/java-archive]
Saving to: `tagsoup-1.2.jar'


100%[===============================>] 90,023 248K/s in 0.4s


2009-10-10 23:19:41 (248 KB/s) - `tagsoup-1.2.jar' saved [90023/90023]

Now invoke this program to fix the HTML file.
$java -jar tagsoup-1.2.jar --nons --files 7777.details.html
src: 7777.details.html dst: 7777.details.xhtml

4. Use xslt transform to transform the xhtml into your desired format.
Create a xsl file to pick data from this xhtml file and create the output. A sample xsl file (names starter.xsl) can be as shown below:

<?xml version="1.0"?>

<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"

version="1.0">

<xsl:template match="/">

<html><head><title><xsl:value-of select="//html/body/table[2]/tr[3]/td[2]/table[1]/tr[3]/td[2]/table[1]/form[1]/input[3]/@value"/></title></head>

<body>

<br/><xsl:text>BugId: </xsl:text>

<xsl:value-of select="//html/body/table[2]/tr[3]/td[2]/table[1]/tr[3]/td[2]/table[1]/form[1]/input[3]/@value"/>

<br/><xsl:text>Summary: </xsl:text>

<xsl:value-of select="//html/body/table[2]/tr[3]/td[2]/table[1]/tr[3]/td[2]/h2"/>

<br/><xsl:text>Resolution: </xsl:text>

<xsl:value-of select="//html/body/table[2]/tr[3]/td[2]/table[1]/tr[3]/td[2]/table[1]/form[1]/tr[3]/td[1]/select/option[@selected='selected']"/>

<br/><xsl:text>Severity: </xsl:text>

<xsl:value-of select="//html/body/table[2]/tr[3]/td[2]/table[1]/tr[3]/td[2]/table[1]/form[1]/tr[3]/td[2]/select/option[@selected='selected']"/>

<br/><xsl:text>Assigned to: </xsl:text>

<xsl:value-of select="//html/body/table[2]/tr[3]/td[2]/table[1]/tr[3]/td[2]/table[1]/form[1]/tr[8]/td[1]/select/option[@selected='selected']"/>

<br/><xsl:text>Priority: </xsl:text>

<xsl:value-of select="//html/body/table[2]/tr[3]/td[2]/table[1]/tr[3]/td[2]/table[1]/form[1]/tr[8]/td[2]/select/option[@selected='selected']"/>

<br/><xsl:text>State: </xsl:text>

<xsl:value-of select="//html/body/table[2]/tr[3]/td[2]/table[1]/tr[3]/td[2]/table[1]/form[1]/tr[9]/td[1]/select/option[@selected='selected']"/>

<br/><xsl:text>Details: </xsl:text>

<xsl:value-of select="//html/body/table[2]/tr[3]/td[2]/table[1]/tr[3]/td[2]/table[1]/form[1]/tr[11]/td[1]/table[1]/tr[1]/td[1]/table[1]/tr[2]/td[1]/pre[1]"/>

</body>

</html>

</xsl:template>

</xsl:stylesheet>

Now you can invoke xsltproc on these files:
$ xsltproc starter.xsl 7777.details.xhtml > 7777.out.html


Notes:
1. It is better to create text files instead of html because line breaks in detailed description are lost unless you escape them properly.
2. You can use lynx -dump 7777.out.html to create a text version of the html file.

XML to XSD in 5 seconds

Recently I created web page to create xsd file for a given xml content. Check it out.

http://sendit.bolona.us/tools/trang/

The UI is not so polished but it does the job.

Joining two mpeg files

Here is a small script to join to mpeg files.


#!/usr/bin/bash

FILE1=$1
FILE2=$2
OUTPUT=OUTPUT.mp2


ffmpeg -i $FILE1 -ss 0 -t 90 -y FILE1.wav
ffmpeg -v 0 -i $FILE1 -ss 0 -t 90 -sameq -y FILE2.m2v

ffmpeg -i $FILE2 -y FILE2.wav
ffmpeg -v 0 -i $FILE2 -sameq -y FILE2.m2v


sox FILE1.wav FILE2.wav all.wav


cat all.wav | mp2enc.exe -o all.mp2
cat FILE1.m2v FILE2.m2v > all.m2v


mplex -V -f 8 -L 48000:1:16 -o all.ps all.m2v $OUTPUT


# if you want to create an mpeg2 transport stream then use
#iso13818ts --fpsi 500 --ps $OUTPUT 1 > OUTPUT.ts

Implementing a Bandwidth Contorlled Pipe in Java

I implemented this one and though will post here so that I can reuse it in future.

import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.OutputStream;
import java.util.Date;


import org.apache.commons.io.output.ThresholdingOutputStream;
public class Pipe extends ThresholdingOutputStream{

long LastTimeTheThresoldHasReached;
OutputStream OStream;

int BWCountPeriodInMiliSeconds;

public Pipe(int bwInBytes,int periodInSec, OutputStream oStream)
{
super(bwInBytes);
OStream =oStream;
BWCountPeriodInMiliSeconds = 1000* periodInSec;
LastTimeTheThresoldHasReached = 0;
}

@Override
protected OutputStream getStream() throws IOException {
return OStream;
}

@Override
protected void thresholdReached() throws IOException {
long now = System.currentTimeMillis();
long timeDiff = now - ( LastTimeTheThresoldHasReached + BWCountPeriodInMiliSeconds);
long bytesWritten = getByteCount();
long bytesExceeded = bytesWritten - getThreshold();
long timeThatWeNeedtoWait = (long) (((float)BWCountPeriodInMiliSeconds / getThreshold() ) * bytesExceeded) -timeDiff;

System.out.print("At " + new Date()+" Waiting for "+ timeThatWeNeedtoWait);

if ( timeThatWeNeedtoWait > 0)
{// We used the allocated quota quicker that we should have. So lets wait.
try {
Thread.sleep(timeThatWeNeedtoWait);
} catch (InterruptedException e) {
}

}

System.out.println("... " + new Date()+" Done");
resetByteCount();
LastTimeTheThresoldHasReached = now;
}



public static void main(String[] args) {

ByteArrayOutputStream baos = new ByteArrayOutputStream();
Pipe pipe = new Pipe(10,10,baos);
while(true)
{
try {
pipe.write("123456789012345".getBytes()); } catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
}

Zoneedit with ddclient on debian

I just thought I should mention this for my own records that this configuration file works for zoneedit+ddclient combination on a debian machine as of 13th April 2009.

pid=/var/run/ddclient.pid,
protocol=zoneedit1,
use=web,
web=http://www.zoneedit.com/checkip.html,
web-skip='Current IP Address:',
server=dynamic.zoneedit.com,
login=MyName,
password='MyPassword',
yourdomain.com,www.yourdomain.com