AUDACITY I KDENLIVE I OPENMEDIA MYPVR 2.0 1 FS-CACHE I SCTP LINUX The SysMaster Tornado M20 packs a media center into a phone Since 1994: The Original Magazine of the Linux C OCTOBER 2007 ISSUE 162 Program the Trolltech Greenphon The State of KDENLIVE OpenMedia myPVR 2.0 FS-Cache for Multimedia Playback Ultimate Linux Home Exploring MPlayer More on Stream Control Transmission Protocol + I Interview with IBM Vice President of Open Source and Qtopia 'X 0-0 4 s hi y pq rs * 5 i kl 8 *uv O O ^ mno www.Unuxjournal.com 1 Q vA/yvz m I HPC Cluster Solutions ^^^ f f| r f it lilllfililim ji[i B I | IjllMlII .Ilf lllllllllllllllll \ 11 "Uisii, Pll!l!!!!ll!!l iniiiimmm I n iji.Ksuj pia fiiiylM fiiiia 111 1 11 in i ii) 11JJ piiiiiiiiiiiiiu ■A r"ll:HI!!!!!l!l pi'Wiiiilittiy Enterprise and High-Performance Computing Under Your Control Appro is by combining the latest technology that meets the demands of the enterprise HPC market. 4-Way XfremeWorkstation 7 AMD Opteron™ 8000 Series processors Up to 128GB of DDR2 533/667 memory Up to 6.0TB SATA or 2.4TB SAS Hot-swappable drives 4-Way 3U Xtreme Server 1 • AMD Opteron™ 8000 Series processors • Up to 128GB of DDR2 533/667 memory • Up to 4.5TB SATA or 1,8TB SAS • 2 PCI-E x16 and 3 PCI-X slots 2 PCI-E xl 6 slots for high- end graphics card Windows® or Linux OS Hot-swappable drives Redundant power supplies & fans ServerDome Management - IPMI 2.0 Windows® or Linux OS For more information, please visit or call Appro Sales at 800-927-5464 AMD Opteron™ • Quad-Core Ready - increase capacity without altering datacenter infrastructure Processors: • Best performance per-watt with energy-efficient DDR2 • Optimized system performance with Direct Connect Architecture Manage Any Data Center. Anytime. Anywhere. Avocent builds hardware and software to access, manage and control any IT asset in your data center, online or offline, keeping it, and your business, “always on”. r „ . The Power of Being There® Visit us on our Remote Control Tour. For locations near you, go to www.avocent.com/remotecontrol. Avocent, the Avocent logo and The Power of Being There, are registered trademarks of Avocent Corporation. ©2007 Avocent Corporation. CONTENTS OCTOBER 2007 Issue 162 FEATURES Open The Tornado M20 Phone and Digital Media Center Is it a PC, media center or phone? Mike Diehl OpenMedia myPVR 2.0 MythTV made easy. Jes Hall Getting Started with the Trolltech Greenphone SDI* beats iPhone. Robert Hartley KDENLIVE Is a Promising Work in Progress KDE-enliven your videos. Dan Sawyer ON THE COVER • The SysMaster Tornado M20. o. 48 • Program the Trolltech Grppnnhonp. n. 56 • The State of KDENLIVE, * >. 62 1 • OpenMedia myPVR 2.0, p. 52 1 • FS-Cache tor Multimedia Playback, p. 90 • Ultimate Linux Home. o. 70 • Exploring MPlayer, p. 84 • More on Stream Control Transmission Protocol, p. 74 • Interview with IBM Vice President of Open Source and Standai ds. Bob Sutor, p. 66 2 | October 2007 www.linuxjournal.com With Coyote Point, you'll never have to wonder if your network is fast enough or flexible enough. You'll know it is. From local to global load balancing, application acceleration or ultimate network manageability. Coyote Point leads the pack. We take the guesswork and difficulty out of application traffic management. You won't find anything faster, smarter or more affordable. Find out why more than 2,000 businesses rely on us to maximize their infrastructure. Learn what Coyote Point could mean to your Web and network traffic. Write info@coyotepoint.com or call 1-877-367-2696. . ■ ’> v * vw. 1* r - X' v. s\ a- *Y .. ' .. <« * 5 f •■''V tBmFT (| -'.I, <\ - v ; ■ *' ■ ^ V .* * . -a,. V • Vv v : ,\,v < -v;*. ?/•■' ■, i \ 4 iLjfflm-, . \ 1 • Ifci ‘J .. 3m. ■ W AHl. i ^ i 14 V < • V s , .A. ..if. > ’ CONTENTS SfS" COLUMNS 20 REUVEN M. LERNER'S AT THE FORGE Incremental Form Submission 28 MARCEL GAGNE'S COOKING WITH LINUX Your Voice, Forever Etched in Electrons INDEPTH 66 INTERVIEW WITH BOB SUITOR Bob Sutor on open source and standards. - G l yn Moody - 70 THE ULTIMATE LINUX HO The Home of the Future awaitf Jon "maddog" Hall 34 DAVE TAYLOR'S WORK THE SHELL Yahtzee as a Shell Script? - When W ill I t End? - 38 JON "MADDOG" HALL'S BEACHHEAD Get FOSSED 40 DOCSEARLS' LINUX FOR SUITS Atlas: Hoisting a New World of Search 96 NICHOLAS PETRELEY'S /VAR/OPINION More Power to Linux IN EVERY ISSUE LETTERS UPFRONT TECH TIPS 46 NEW PRODUCTS 81 ADVERTISERS INDEX 74 STREAM CONTROL TRANSMISSION PROTOCOL (SCTP) ASSOCIATIONS Associations and connections. Jan Newmarch 90 FS-CACHE AND FUSE FOf MEDIA PLAYBACK QOS Playback smoothing. - Ben Ma r l i n - USPS LINUX JOURNAL (ISSN 1075-3583) is published monthly by Belltown Media, Inc., 2211 Norfolk, Ste 514, Houston, TX 77098 USA. Periodicals postage paid at Houston, Texas and at additional mailing offices. Cover price is $5.99 US. Subscription rate is $25/year in the United States, $32 in Canada and Mexico, $62 elsewhere. POSTMASTER: Please send address changes to Linux Journal, PO Box 980985, Houston, TX 77098. Subscriptions start with the next issue. Next Month HIGH-PERFORMANCE COMPUTING The power of Linux clusters and high-performance computing is undeniable, and we've got num¬ bers to prove it. We also show you how to create a high-avail¬ ability solution with Red Hat Enterprise Linux. Don't have a roomful of computers handy? You can still leverage the HPC capabilities of Linux by using distcc to distribute your compile tasks across the few computers you happen to have running. As always, there's much more. How well do you think Ubuntu installer will do when put through a usability test? We've got the results. Who needs a PIM when you've got Emacs? We'll show you how to make this edi¬ tor keep you organized. 4 | October 2007 www.linuxjournal.com The ISP Solution of Choice for a Carrier Class Mail Server CLUSTERING SYSTEM High Availability (RHCS integration) Distributed mailboxes (LDAP routing) ■ SECURITY & CONTROL Antivirus & Antispam integration Reporting engine (over 100 reports) Online Backup & Restore system POWER FEATURES Delegated Administration Account Classes Personal Organizer Unlimited Servers www.axigenxom/Lj LINUX JOURNAL Editor in Chief Nick Petreley, ljeditor@linuxjournal.com Executive Editor Jill Franklin jill@linuxjournal.com Senior Editor Doc Searls doc@linuxjournal.com Art Director Garrick Antikajian garrick@linuxjournal.com Products Editor James Gray newproducts@linuxjournal.com Editor Emeritus Don Marti dmarti@linuxjournal.com Technical Editor Michael Baxter mab@cruzio.com Senior Columnist Reuven Lerner reuven@lerner.co.il Chef Fran^ais Marcel Gagne mggagne@salmar.com Security Editor Mick Bauer mick@visi.com Contributing Editors David A. Bandel • Ibrahim Haddad • Robert Love • Zack Brown • Dave Phillips Marco Fioretti • Ludovic Marcotte • Paul Barry • Paul McKenney • Dave Taylor Proofreader Geri Gale Publisher Carlie Fairchild publisher@linuxjournal.com General Manager Rebecca Cassity rebecca@linuxjournal.com Director of Sales Regional Sales Manager Regional Sales Manager Laura Whiteman laura@linuxjournal.com Joseph Krack joseph@linuxjournal.com Kathleen Boyle kathleen@linuxjournal.com Circulation Director Marketing Coordinator Mark Irgang mark@linuxjournal.com Lana Newlander mktg@linuxjournal.com System Administrator Webmaster Mitch Frazier sysadm@linuxjournal.com Katherine Druckman webmaster@linuxjournal.com Accountant Candy Beauchamp acct@linuxjournal.com Linux Journal is published by, and is a registered trade name of, Belltown Media, Inc. PO Box 980985, Houston, TX 77098 USA Editorial Advisory Board Daniel Frye, Director, IBM Linux Technology Center Jon "maddog" Hall, President, Linux International Lawrence Lessig, Professor of Law, Stanford University Ransom Love, Director of Strategic Relationships, Family and Church History Department, Church of Jesus Christ of Latter-day Saints Sam Ockman, CEO, Penguin Computing Bruce Perens Bdale Garbee, Linux CTO, HP Danese Cooper, Open Source Diva, Intel Corporation Advertising E-MAIL: ads@linuxjournal.com URL: www.linuxjournal.com/advertising PHONE: +1 713-344-1956 ext. 2 Subscriptions E-MAIL: subs@linuxjournal.com URL: www.linuxjournal.com/subscribe PHONE: +1 713-589-3503 FAX: +1 713-589-2677 TOLL-FREE: 1-888-66-LINUX MAIL: PO Box 980985, Houston, TX 77098 USA Please allow 4-6 weeks for processing address changes and orders PRINTED IN USA LINUX is a registered trademark of Linus Torvalds. AMD Power of Processors of Memory HPC Systems, Inc. 48009 Fremont Blvd Fremont, CA 94538 Tel: 510-656-8282 Fax: 510-656-8341 E-mail: info@hpcsystems.com Based on industry leading AMD Opteron™ microprocessors, the HPC A5808-32 server provides industry leading scalable x64 processing with the latest I/O, network, memory, and power efficient technologies. With features such as AMD quad-core Opteron™ microprocessors and up to 256GB of DDR2 for memory-intensive performance, multithreaded applications like CRM, ERP, e-commerce, and virtualization will see significant performance improvements. And HPC MasterSight™ delivers features to help manage the system with inclusive diagnostic tools. Specializing in High Performance Computers, HPC systems, Inc, is a solution provider saving the financial, health, educational, and government sector. We appreciate that you have a choice of dozens of vendors, but not all of them have our uncompromising dedication and experience in producing the best solutions. AMDtl Solution Provider PLATINUM GSA GS-35F-0596R Contract Holder AMD, AMD Opteron. end flna AMD logo' rggstered trgdemalra of the Advanced Oewee or He sufcsicijies in the US end oth? ocMTtroe. letters A DJANGO QPEN5WAN MAMUHL PLAYSTATIONS ICECA5I Iwa Jind Icocnst SlflKWTlMlfl Auto ennw*Jofii UnuK SOP Jnlra to Inotail, ‘®X£0 t* % ‘Hi *Jp * \ \ Vl Hcxt'Q[?ncratHin {tewMisI Gateway BUILD IT YOURSELF ARCADE AND OTHER COOL PROJECTS NOKFA ESI idiic QPENSWAN VPN PLAYSTATION 3 SUPER COM PUTINS CLOST^ft KAGNATUNE re MOT EVIL 1^ EmKM ■ Fiffliox E^lsnwns 64-flilUihi* Inaccurate Items in Magnatune Article I have to raise an objection (or five) to the Magnatune article in the August 2007 Linux Journal. First, Apple's iTunes uses AAC, the same audio codec for DVDs you get off the shelf. This codec is much tighter in compression but has better audio quality than MP3. Second, DRM'd iTunes music is encoded at 128kbps using AAC. A buck a song at 128kbps sounds rather decent. Third, EMI's non-DRM'd music will sell for $1.29, but it will be double the encoding rate, 256kbps, so it will sound much better. This doubling of the bandwidth (which I doubt will all be used) will make the files pulled bigger, and thus the extra cost is justifiable. Fourth, the non-DRM'd music is still encoded in AAC format and possibly wrapped in an MPEG-4 container. It won't be MP3'd; it won't be WMA'd—both for Apple's legal eye will require licensing the codec. Plus, iPods can't listen to Ogg Vorbis files either. Apple hasn't thought about putting in the free (as in freedom) codec in the iPod's firmware as well as QuickTime. Fifth, EMI's music catalog isn't all DRM-free yet. Apparently, the person who wrote the article forgot to do his research. I believe the keynotes of the past WWDC and MacWorld conferences should help. They're still up on iTunes as podcasts for free (as in beer). Kelly "STrRedWolf 11 Price Sold on Kubuntu Being a UNIX/Linux user for more than 30 years and seeing the evolution of computers has been very interesting. First, we had large pieces of iron on raised floors and huge air conditioning bills and large support staff, and we all carried large boxes of cards for the larger programs. I got my PhD in Astrophysics in 1972, and my dissertation required more than $1,000,000 in computer time on a CDC 7600 and then later on a Cray IS. It was taking 3.5 hours per case for simulation of light transport in real planetary atmospheres. Now I can do a case in less than ten minutes on an AMD64. I still wonder if people fully realize just how much compute power they have at their fingertips. I wrote a book in 1978, and in Chapter 1, I wrote that I had figured out what the human race was doing overall. We are trying to get all of human knowledge at the fingertips of every man, woman and child on the planet. We are about to get there. Back to your article [Nicholas Petreley's "Amazing Free Distributions Abound", July 2007], I used SUSE for years and years. But last year, I was converted to Kubuntu, and with 7.04, there is no way I can ever go to another distro. It has, with all the official repositories, given me access to more programs and utilities than I'll ever need. Last year, I developed a short Intro to Linux course for free adult training centers and continuing education facilities. I use Kubuntu 7.04 because of the live CD and the ease of installation. I also use a cheap Airlink USB Wi-Fi critter for $10 on sale at Fry's Electronics that plugs and plays without intervention by the owner and really, really helps get things going. I am working on getting freshmen high- school students to start up on Linux. Think about it. They would never have to buy another piece of software for the rest of their high-school and college careers. They'd have every compiler they ever would need. They would have OpenOffice.org for their reports and spreadsheets, and LaTeX for engineers and mathematicians for any mathematics they encounter. The list goes on and on. I developed and taught all the courses at Silicon Graphics until I retired 12 years ago, when SGI started downhill. So, I guess it's still in my blood to continue the fight against ignorance. "Push back the frontiers of ignorance" was my motto as a prof and instructor. Get a thought to go where no thought has gone before. Keep up the good work. Chuck Adams Don't Bogart That Link, My Friend In the Tech Tips column in the July 2007 issue, a method to prevent services from starting is described for those who do not want to use the graphical interfaces. However, there is a specific command-line tool available just for that task, chkconfig. I think this tool works even better than crowding the /etc/rc[0-6].d directories with unclear links. From the chkconfig man page: "chkconfig provides a simple command-line tool for maintaining the /etc/rc[0-6].d directory hierarchy by relieving system administrators of the task of directly manipulating the numerous symbolic links in those directories." To follow the example in the magazine, instead of renaming S25bluetooth to s25bluetooth, simply type chkconfig bluetooth off (and chkconfig bluetooth on to turn it back on again). Stefan Roels 8 | October 2007 www.linuxjournal.com MORE PRODUCTS, BETTER SERVICE, GUARANTEED. [EEP Yl GO STRAIGHT TO THE SOURCE! IESS RUNNING SMOOTHLY PROTECT YOUR SMALL BUSINESS WITH THE BUILT-IN SECURITY ENHANCEMENTS OF THE DUAL-CORE INTEL® XEON® PROCESSOR IN YOUR SERVERSDIRECT SYSTEM. SDR-1105T1U ENTRY LEVEL SERVER EXCELLENT GENERAL PURPOSE SERVER FOR ORGANIZATIONS WITH THE NEED FOR A LOW, ENTRY LEVEL PRICE * 1U Rackmount Chassis with 520W power supply * Supermicro X7DVL-L Server Board with Intel® 5000V (Blackford VS) Chipset * Intel® Dual-Core Xeon Processor 5050 3.0GHZ 667 MHz * Total 512MB, 2pcs x 256MB Kingston DDR2 533Mhz FB-DIMM ECC * Seagate SATAII 80GB 7200 RPM 8MB Cache SATA 3.0Gb/s Hard Drive *4x1” Hot-swap SATA Drive Bays * Two Intel® 82563EB Dual-port Gigabit Ethernet Controller * Intel® ESB2 SATA 3.0Gbps Controller RAID 0,1, 5,10 support STARTING PRICE SDR-Z503T ZD APPLICATION SERVER Highest performing with Dual Core/ Quad Core Xeon CPU based. Excellent with general purpose applications and provide the most power. * 2U Rackmount Chassis with 650W power supply * Seagate SATAII 250GB 7200 RPM 8MB Cache SATA 3.0Gb/s * Supermicro X7DVL-E Server Board with Intel® 5000V (Blackford VS) Hard Drive Chipset *6x1” Hot-swap SATA Drive Bays * Intel® Dual-Core Xeon Processor 5050 3.0GHZ 667 MHz * Intel® (ESB2/Gilgal) 82563EB Dual-port Gigabit Ethernet Controller * Total 512MB, 2pcs x 256MB Kingston DDR2 533Mhz FB-DIMM ECC * Intel® ESB2 SATA 3.0Gbps Controller RAID 0, 1,5, 10 support SDR-3500T 31) DATABASE SERVER Easily Scalable storage solution with hot-swap functionality for growing businesses STARTING PRICE $2,199 * 3U Rackmount chassis with Redundant 800W power supply * Supermicro X7DBE+ Server Board with Intel® 5000P (Blackford) Chipset * Intel Quad-Core Xeon Processor E5310 1.6GHZ " Total 1024MB, 2pcs x 512MB Kingston DDR2 533MHz FB-DIMM ECC Seagate SATAII 500GB 7200 RPM 16MB Cache SATA 3.0Gb/s Hard Drive 16x1" Hot-swap SATA Drive Bays Dual-port Gigabit Ethernet Controller Intel ESB2 SATA3.0Gbps Controller RAID 0, 1, 10 support SDR-7045B-TB40 FILE SERVER 4U Quad-Core Xeon Server offers excellent value and expandability "■ 4U Rackmountable / Tower with 650W power supply * Supermicro Super X7DBE Server Board with Intel® 5000P (Blackford) Chipset * Intel Quad-Core Xeon Processor E5310 1.6GHZ * Total 1024MB, 2pcs x 512MB Kingston DDR2 667MHz FB-DIMM ECC Seagate SATAII 750GB 7200 RPM 16MB Cache SATA 3.0Gb/s Hard Drive 6 x 3.5" Hot-swap SAS/SATA Drive Bays Dual-port Gigabit Ethernet Controller Intel ESB2 SATA3.0Gbps Controller RAID 0, 1,5, 10 support SDR-5111T 5B ADVANCED STORAGE SERVER Quad Core dual Xeon CPU based, with 24 hot-swap hard disk bays suitable for 18TB of pure data Storage capacity STARTING PRICE * 5U Rackmount chassis with Redundant 1350W power supply * Supermicro X7DBE Server Board with Intel® 5000P (Blackford) Chipset * Intel Quad-Core Xeon Processor E5310 1.6GHZ * Total 1024MB, 2pcs x 512MB Kingston DDR2 667MHz FB- DIMM ECC c Seagate 750GB 7200 RPM 16MB Cache SATA 3.0Gb/s Hard Drive c 24 x 1" Hot-swap Drive Bays f Intel® (ESB2/Gilgal) 82563EB Dual¬ port Gigabit Ethernet Controller f Intel ESB2 SATA 3.0Gbps Controller RAID 0, 1,5, 10 support SERVERS DIRECT CAN HELP YOU CONFIGURE YOUR NEXT HIGH PERFORMANCE SERVER SYSTEM - CALL US TODAY! Our flexible on-line products configurator allows you to source a custom solution, or call and our product experts are standing by to help you assemble systems that require a little extra. Servers Direct - your direct source for scalable, cost effective server solutions. 1.877.727.7887 | www.ServersDirect.com Intel, Intel logo, Intel Inside, Intel Inside logo, Intel Centrino, Intel Centrino logo, Celeron, Intel Xeon, Intel SpeedStep, Itanium, Pentium, and Pentium III Xeon are trademarks of Intel Corporation or it’s subsidiaries in the United States and other countries. Xeon* inside ™ Dual-core. Do more. [LETTERS] Don't Short Python It's a little late to be writing about the June 2007 issue, but I just reread the Lua article, and I'm a bit disappointed, so I thought I'd write. In general, it seems to be a pretty good introduction to Lua. However, the author did a lot of comparing to Python, and I think that comparison was done unfairly. Whenever such a language article is writ¬ ten, it really should be reviewed by an expert in the language it's being com¬ pared to. The author is clearly not very experienced with Python, and it shows. Lua has a key advantage for its intended purpose (its impressively small size), but people should select it for its real advantages, not because of mistaken impressions. In general, it looks to me like many of Lua's traits make it quite unsuitable for large programs, which is fine, as it's not intended for that. However, Python is pretty darn suitable for both large and small. Chris More Ruby on Rails I have really been digging Linux Journal ever since I subscribed to your fine pub¬ lication. Lately, I've been enjoying the shell scripting articles, and in the July 2007 issue, the article on vector graph¬ ics was rather interesting. In the future, I think you guys should do some more articles on Ruby and Ruby on Rails. I really enjoyed the one edition almost completely dedicated to Ruby. It would really tickle my fancy if you put some more Linux Web-development-related things into the magazine. Anyway, keep up the good work with making my favorite Linux publication. Patrick No SOP Here, Move Along As a developer of embedded systems, including Linux, for the past two decades, I was very interested in the article "Standard Operating Procedures for Embedded Linux Systems" [August 2007], The article seems to describe a limited approach to embedded Linux develop¬ ment and certainly not an "SOP". The primary software described, buildroot, is a terrific and powerful tool for embed¬ ded developers, as are many packages from uclibc.com; however, such high-level tools come with limitations. Buildroot, for example, is heavily entwined with the uClibc library, which is not a viable choice for every system. The "five standard pro¬ cedures" violate basic design methodology by selecting the hardware first. Once all packages are selected, only then can a tool like buildroot be considered; only amateurs select the tools before determining the task. The article appropriately focuses on reducing memory and storage require¬ ments, but no mention is made of alter¬ nate compressed filesystems, Flash filesystems and especially the new xip (execute in place) file support. Understanding the limitations of the space-saving software, such as uClibc and BusyBox, requires deep knowledge but is ignored in this article. Using packages that are not designed for cross-build is very difficult, despite the article's statement that it sometimes makes good sense to build these packages on the target. Bootup and shutdown time is important for many embedded systems, and this has design implications regarding hardware and software but isn't men¬ tioned. Often, some new kernel feature is required, but upgrading the kernel may not be possible (many BSPs are dropped or rendered nonfunctional in time), so devising a solution is difficult, which is not addressed. I wish we could download a tool, run a configurator and have a functional image pop out, trim a little "fat", install and ship, but life is rarely so simple. It's a nice article on one group's basic development process, but there is nothing standard here. "One size fits all" isn't yet a viable approach for embedded Linux. Steve Alexander LINUX JOURNAL fit Your Service MAGAZINE PRINT SUBSCRIPTIONS: Renewing your subscription, changing your address, paying your invoice, viewing your account details or other subscription inquiries can instantly be done on-line, www.linuxjournal.com/subs. Alternatively, within the U.S. and Canada, you may call us toll-free 1-888-66-LINUX (54689), or internationally +1-713-589-2677. E-mail us at subs@linuxjournal.com or reach us via postal mail, Linux Journal, PO Box 980985, Houston, TX 77098-0985 USA. Please remember to include your complete name and address when contacting us. DIGITAL SUBSCRIPTIONS: Digital subscriptions of Linux Journal are now available and delivered as PDFs anywhere in the world for one low cost. Visit www.linuxjournal.com/digital for more information or use the contact information above for any digital magazine customer service inquiries. LETTERS TO THE EDITOR: We welcome your letters and encourage you to submit them to ljeditor@linuxjournal.com or mail them to Linux Journal, 1752 NW Market Street, #200, Seattle, WA 98107 USA. Letters may be edited for space and clarity. WRITING FOR US: We always are looking for contributed articles, tutorials and real- world stories for the magazine. An author's guide, a list of topics and due dates can be found on-line, www.linuxjournal.com/author. ADVERTISING: Linux Journal is a great resource for readers and advertisers alike. Request a media kit, view our current editorial calendar and advertising due dates, or learn more about other advertising and marketing opportunities by visiting us on-line, www.linuxjournal.com/advertising. Contact us directly for further information, ads@linuxjournal.com or +1 713-344-1956 ext. 2. ON-LINE WEB SITE: Read exclusive on-line-only content on Linux Journal's Web site, www.linuxjournal.com. Also, select articles from the print magazine are available on-line. Magazine subscribers, digital or print, receive full access to issue archives; please contact Customer Service for further information, subs@linuxjournal.com. FREE e-NEWSLETTERS: Each week, Linux Journal editors will tell you what's hot in the world of Linux. Receive late-breaking news, technical tips and tricks, and links to in-depth stories featured on www.linuxjournal.com. Subscribe for free today, www.linuxjournal.com/enewsletters. V 10 | October 2007 www.linuxjournal.com Maximize AMD Opteron™ Quad-Core Performance with UniServer to 32GB memory 1U UniServer with 1 Socket • Dual/Quad-Core AMD Opteron™ 2000 series • Up to 32GB of memory • 2 hot swap SATA 3.5" HDDs • MCP55V Pro chipset • Slim CD/DVD-ROM • 1 PCI-Express x16 slot • Remote management-IPMI 2.0 11) Server: 1 & 2 Sockets, up to 64GB memory 2U Server: 2 & 4 Sockets, up to 128GB memory 3U Server: 2 & 4 Sockets, up to 128GB memory Workstation: 1,2 & 4 Sockets, up to 128GB memory Bell Micro www.bellmicro.com 1-800-291-2070 www.avnet.com Applied Computing Solutions 1-888-300-8277 Q SYNNEX www.synnex.com 1-888-756-4888 www.uniwide.com 1-877-520-0071 Uniwide is an official AMD Validated Server Program Partner Copyright © 2007 Uniwide Technologies, Inc. All right reserved. Specifications are subject to change without notice. AMDil Opteron FRONT NEWS + FUN Ondrej Zary has produced a new free software driver to support IRTouchSystems touchscreens. The manufacturer releases only closed-source drivers, so Ondrej had to hack into the data flow itself and do his own analysis in order to understand how to interact with the hardware. There's a new performance monitoring tool on the block. Mark Seger has been coding and tweaking his collectl tool (collectl.sf.net) for about four years, and he finally has decided to release it under a free software license. It tracks a whole bunch of system stats and displays them in various formats. Mark has invited everyone to use it, abuse it and send him bug reports. Although because it already has been in use on his personal system for years, it probably is quite robust and stable. Intel's coming out with some new IOMMU hardware, aka Virtualization Technology for Directed I/O, and Anil S. Keshavamurthy has posted a patch to support that hardware in Linux. There is still some question as to whether the code is quite ready to be integrated into the ker¬ nel— Andrew Morton thinks it might be too slow—but overall, Anil's work has received plenty of support, and it is excel¬ lent to see Intel being so proactive in sup¬ porting its hardware. Chris Mason has been working on BTRFS (BTRee filesystem), a new effort aimed at data integrity. As Albert D. Cahalan said during the discussion follow¬ ing Chris's announcement, storage hard¬ ware can't really be trusted, so filesystems have to take up the slack and make sure users' data doesn't get lost. BTRFS seems to be finding a lot of support among kernel developers; however, it's still in the early stages and shouldn't be used for any kind of production environment yet. One of Chris' main ideas is to keep data integrity support simple, even if it means not implementing fancier features that may be popular among other filesystems. Every once in a while someone decides to dig out one of the particularly ugly parts of the kernel and fix it up nice. Nick Piggin has done that recently with the buffer layer. This is the part of the kernel that deals with block devices, tracking a variety of data, including the status of writes to particular disk blocks. Nick's replacement, fsblock, is a much cleaner, shinier implementation. Among other improvements, fsblock's support for large block sizes is much simpler and nicer than what was there before. His code also avoids the deadlocks common to the existing version. But, although these and many other improvements definitely are great, fsblock is still in the early stages of development, and folks like Jeff Garzik are concerned that other aspects of the buffer-layer problem space may force fsblock's code into nastier and more tangled knots. So far, the only filesystem Nick has ported to use fsblock has been Minix, a famously simple and academic filesystem intended for student consumption. The more complicated journaling filesystems, such as ext3/4 and ReiserFS, will be the real test. And, even if fsblock eventually does rise to that challenge, developers like Christoph Hellwig don't think Nick's improvements are significant enough to justify replacing the whole buffer layer and porting all that filesystem code. — ZACK BROWN diff -u WHAT'S NEW IN KERNEL DEVELOPMENT USEH FRIENDLY by J.D. llllitf' Fruit _ m wm Eimii WE WHO BEAR THE BURDEN Of KU0WLEDSE HEAR YOUR £R* T££H SUPPORT sres spe wm. \ I HAVE A DVD I \ WAUT TO PLAY \ WW LJHUX BOX. jr t THE PROBLEM IS THAT i XIUE SEEMS TO FLA/ g IT BACK ALL CROPPY * AMD STUFF: V, THAT'S JUST = ^ DMA CHECKUP 1 i SO HOW DO I MAKE IT SO I £AfJ WATOH M/ DVD? V BLWK eves \ SVMOHED TO THE \ DMA OHEtrtlKte. A THAMKS FOR / CMJLM&l i LJ Index, October 2007 1. How many dollars more a Linux-based Dell Inspiron 1420 cost over the same one running Vista, before the mistake was corrected: 225 2. How many dollars less a Linux-based Dell Inspiron 1420 cost under the same one running Vista, after the mistake was corrected: 50 3. Corrected base price in dollars for the Dell 1420 Inspiron notebook running Ubuntu: 774 4. Number of case color choices for the 1420: 8 5. Percentage of computers refurbished by the Alameda County Recycling Center (ACCRC) that run Linux: 100 6. Price charged by ACCRC for taking in any computer: 0 7. Price charged by ACCRC for taking "anything that you can plug in to a power outlet" other than large noncomputing appliances and monitors and TVs with no source: 0 8. Price in cents charged per pound for taking in monitors and TVs with no source: 50 9. Thousands of supported Linux systems given away per year by the ACCRC: 1 10. Thousands of new Apache sites found by Netcraft for its July 2007 survey: 556 11. Thousands of new Google sites found by Netcraft for its July 2007 survey: 592 12. Millions of new Microsoft sites found by Netcraft for its July 2007 survey: 2.4 13. Apache's percentage share of all active Web sites surveyed by Netcraft: 49.98 14. Microsoft's percentage share of all active Web sites surveyed by Netcraft: 35.48 15. Google's percentage share of all active Web sites surveyed by Netcraft: 4.35 16. Google's percentage share of all active Web sites surveyed by Netcraft in July 2006: 0 17. Apache percentage advantage over Microsoft in July 2006: 33.4 18. Apache percentage advantage over Microsoft in July 2007: 14.5 19. Percentage of North American developers targeting Linux in 2007: 11.8 20. Percentage increase of the above over 2006: 34 Sources: 1-3; 18-20: DesktopLinux.com 4-7: APPC (www.accrc.org) | 8: ZaReason, Inc. 9-17: Netcraft.com 12 | October 2007 www.linuxjournal.com AMD They Said It When something that was originally scarce starts becom¬ ing abundant, something strange happens. You find that you start making money because of that thing rather than with that thing. That's the Because Effect. —JP Rangaswami, British Telecom, confusedofcalcutta.com/ 2007/07/08/prince-ly-returns-from-the-because-effect Non-open-source users typically use price as a factor in their decisions. They think that a more expensive com¬ puter will be faster, more sturdy, and will last longer. That's not necessarily true. It is definitely not true in software, and it is becoming less true in hardware....We believe that all our other worthy goals (such as gaining market share for Linux) will be accomplished best by a myopic, utterly self-absorbed focus on solid hardware and honest customer support. —Cathy Malmrose, CEO, ZaReason, Inc., allaboutubuntu.wordpress.com/2007/06/2 6/zareasons-ceo/ #more-62 Today Gates openly concedes that tolerating piracy turned out to be Microsoft's best long-term strategy. That's why Windows is used on an estimated 90% of China's 120 million PCs. "It's easier for our software to compete with Linux when there's piracy than when there's not", Gates says. "Are you kidding? You can get the real thing, and you get the same price." Indeed, in China's back alleys, Linux often costs more than Windows because it requires more disks. And Microsoft's own prices have dropped so low it now sells a $3 package of Windows and Office to students. —David Kirkpatrick, in Fortune , money.cnn.com/magazines/ fortune/fortune_archive/2007/07/23/100134488/index, htm Bottom line—this was an oversight, pure and simple. —Dell spokesperson, on why the Linux-equipped Inspiron 1420s at first appeared to cost more than Vista-equipped ones, www.desktoplinux.com/news/NS9933912441 .html It's clear that a shift away from Windows began about two years ago, and the data show that this migration is now accelerating. Linux has benefited, but we also see corresponding growth in niche operating systems for nontraditional client devices. The landscape is changing. —John Andrews, Evans Data Corp., www.evansdata.com/press/viewRelease. php?presslD=51 None of this technology is optimally applied to getting laid. —Jeff Waugh, talk at Guadec 2007Z ,_ J\ Expert Included. Opteron Tim is dedicated to processes that make every server from Silicon Mechanics a model of consistency and reliability. The build and quality processes he applies guarantee that your server arrives ready to perform. He likes servers that combine outstanding performance with long-term value. The Rackform nServ A411 is impressive because it's a compute cluster in one rack unit, with four Dual-Core AMD Opteron™ 8000 Series processors and 16 DDR2 DIMM sockets. The Direct Connect Architecture allows all eight cores to work together with maximum efficiency. Tim knows that the 4 Dual-Core Second-Generation processors are designed to offer seamless upgradeability to Quad-Core. When you partner with Silicon Mechanics, you get more than a powerful AMD server—you get an expert like Tim. MECHANICS visit us at www.siliconmechanics.com or call us toll free at 866-352-1173 Silicon Mechanics and the Silicon Mechanics logo are registered trademarks of Silicon Mechanics, Inc. AMD, the AMD Arrow logo, AMD Opteron, and combinations thereof, are trademarks of Advanced Micro Devices, Inc. [UPFRONT Ten Questions for Sean Moss-Pultz When iPhone launched in late June 2007, Newsweek snarked, "Not since Moses parted the Red Sea has anything this miraculous appeared on earth. Will it cure cancer?" Meanwhile, the OpenMoko Project has been hard at work providing earth with a Linux-based cure for iPhone. We've covered OpenMoko a number of times already, but all were before iPhone parted the seas of mainstream ink otherwise being spilled on war, politics and Paris Hilton coverage. So, we inter¬ rupted the overbooked time of Sean Moss-Pultz, Program Manager of OpenMoko, to probe his thinking about OpenMoko in a post-iPhone world. LJ: First, can you give us a point-by-point comparison of OpenMoko with iPhone? SM-P: You can check this out: wiki.openmoko.org/wiki/IPhone. LJ: But really, OpenMoko is not an "open-source alternative" to iPhone? SM-P: No, not even close. We're in a totally different market. The last thing on earth I want to do is start copying the iPhone—no offense to Apple. They've built a beautiful product, but do we really need another closed, locked-down phone? Fundamentally, we're totally different. End-user Freedom is our passion. Apple is about giving you an incredibly polished experience—exactly how they want you to have it. End users really have no freedom. They cannot change the device if they don't like the way Apple chooses to make things. OpenMoko is the anti-iPhone. LJ: So, you're pro-FOSS? SM-P: The entire OpenMoko system and application software are built using free and open-source software (FOSS). On the system side, OpenMoko uses software with a tireless history of success and stability, such as the Linux kernel, the GNU C library, the X Window System and the GTK+ toolkit, to name only a few. OpenMoko is Mobile FOSS. Apple won't even give you an SDK for the iPhone. We give you everything for Free—exactly the same tools that we use internally. We want you to change this device—personalize away to your heart's content. I like to describe OpenMoko as a movement to create an open platform that empowers people to personalize their phones, much like computers, in any way they see fit. Apple makes sure their entire software stack stays closed. We chose to make the entire software stack open. From a control standpoint—the things corporations love— this borders on insanity. But, I think by pushing these borders, we will let loose the possibility for immense innovation. Innovation, in my opinion, is seldom found within the endless cubicles of a large corporation. Most commonly, it manifests itself within the intense focus and concentration that all individuals seem to have access to when they stare at a single problem long enough. Staying with a problem long after most would quit is a luxury few companies can afford. Instead, I want to focus on the fundamentals—the framework—to use a more specific term. This includes the following parts: ■ Ul—common look and feel for end users. ■ Data—common storage model for applications. ■ Libraries—common platform for developers. We believe these are some of the key areas to solidify for innovation to form. And, we believe this will benefit not just my company (FIC), but everyone who uses a mobile phone. We really try to make things as easy as possible for developers. Software-wise, our platform is still in the early stages, but things are moving fast. It's an incredibly exciting time for us now. LJ: Does OpenMoko see verticals as the key to breaking open the twin silos of phone makers and carriers? If so, which ones? You mention "major corporations. Fortune 500s, hospitals, real estate, engineering, the arts...." Is OpenMoko downstream with any of those? How? SM-P: Enterprise is a very interesting vertical market for us now. I can't mention exactly whom we're talking with currently, but it's all the big names. If you think about this, it makes total sense. They have huge IT budgets and the ability to put scores of people working on custom solutions. More and more they also are running GNU/Linux on the server. So, having another client (besides cumbersome PCs) makes for quite a value proposition. LJ: Here's why we ask. Steve Jobs said the iPhone is closed to developers at Cingular's (now AT&T's) request. Companies like AT&T generally don't like open phones. Yet, we have evidence that phone makers and carriers will both open up and make exceptions for big vertical buys. SM-P: Yeah, I think this will be a huge ZaReason It's Good In the last couple issues' UpFront sections, we've followed Dell's toe-dipping into the surging demand for Linux-based laptops and desktops. It's important, however, to look at the other end of the corporate scale for gear sources as well. EmperorLinux and LinuxCertified are two familiar standouts there. A new one starting to get attention is ZaReason, Inc., in Berkeley, California. ZaReason grew out of family volunteer work at the Alameda Computer Recycling Center (see the LJ Index this month for some interesting stats about the ACRC) and crafts cheap, high-quality machines, intended mostly for hands-on techies. "Typically, we ship to longtime Linux activists who want fully open-source machines, but do not have the time to research components and configurations", says the company's About page. Check 'em out: www.zareason.com/shop/home.php, — DOC SEARLS 14 | October 2007 www.linuxjournal.com market. I mean, most everyone I know in business uses company phones. Why not let the company personalize the phone for its employees? LJ: Who are the early developers, exactly? SM-P: Hackers, developers, technologies, hobbyists...students, really, we've seen so many people it's mind-boggling. I'm totally blown away by the sheer volume of interest we are getting now. This is quite a pleasant surprise. Sure, we thought an open phone would shake things up a bit. But this is more like an earthquake. LJ: What are we seeing in pickup from different parts of the world? SM-P: Well, I would say it's still mainly a western thing. Europe and the US represent most of the e-mails. But, South America, Africa and Asia now are really starting to pick up pace. It's really getting global! LJ: Jonathan Schwartz of Sun showed off an OpenMoko phone, no? What more is coming of that? SM-P: Yeah, that was our Neo. I really can't make any more comments than that now. Sorry. LJ: How do you see the open-phone market developing? In what ways will it resemble and/or differ from the "white box" computer market in which Linux grew? SM-P: Oh, great question. I think the market will be quite different. In the embedded system world, there is just a tighter level of integration between hardware and software. LJ: What about other phone uses, such as camera, texting, audio (for example, podcast) recording and music/video playback? SM-P: We need to open the mobile ecosys¬ tem. A mobile phone has the potential to be a platform that can do anything that a small computer with broadband access can do. For the people pushing this project, an open phone is not really even a product. It's the very embodiment of our vision of tech¬ nology. We absolutely, passionately, believe that something as fundamental to our lives as the mobile phone must be open. — DOC SEARLS Your World Runs c-tree* Database Faster With Technology A small footprint c-tree database controls the traf f ic lights on your way to work. Packages are scanned and delivered using a scalable c-tree database that can run on handheld devices and mainframes. Your f inancial transactions are secure because they are authenticated using a c-tree database. A high throughput c-tree database validates your credit card transactions. Your world runs faster (and you sleep better!) with c-tree database technology. Your digital pictures are well-organized thanks to the transparent deployment of a c-tree database within your photo album software. FairCom provides high performance, low maintenance data management technology. Our customers - ranging from small startups to multinational corporations - are able to overcome application-specific performance dilemmas because c-tree gives them precise control over their database operations. Super-charge your application and simplify your deployment! Download an evaluation edition of c-tree today. 0 FairCom Download FairCom’s c-tree Plus Today! www.faircom.com/go/7usesDLD High Performance Database Technology • 800-S34-8180 Other company and product names are registered trademarks or trademarks of their respective owners. © 2007 FairCom Corporation [UPFRONT A Ticket out of Poverty Looking beyond the walls of our temporary office in Nairobi, Kenya, one can see fences with barbed wire and security guards near any building audacious enough to show a sign of wealth. Everything seems brown here, from the dust on the roads and in the air to the rusty tin roofs protecting most residents from the sun and rain. We are in Kibera—with an estimated 1.5 million residents, it is the largest slum in Africa. Our project is unconventional, for not only do we avoid hiring security guards, we also were invited by the youth within the community. We find ourselves here to run a media development project, powered by open-source software. We avoided most travel warnings and, to some, left our com¬ mon sense at the border when we chose to bring laptops and digital cameras to one of the poorest areas in the world. Many resi¬ dents in Kibera live on a few dollars a day, grappling problems like HIV/AIDS and mal¬ nutrition, and have never used a computer before. When we arrived, however, we were greeted by quick-learning youth interested in trying new technologies and media tools to improve their lives and the state of their entire community. This is the second such project that Five Minutes to Midnight (FMM, www.fiveminutestomidnight.org), a youth-led organization from Canada, is running. Initially started to promote youth involvement in human rights through media and journalism, FMM launched the Article 13 Initiative (A131, www.a13i.org) in December 2005 to promote the use of open-source software in media projects in developing countries. Its pilot project took place that December and the following January in N'Djamena, Chad, one of the poorest countries in the world. With a base operating system of Fedora Core and packages, such as The GIMP, Inkscape and OpenOffice.org, the project began through a partnership with Rafigui, Chad's only national youth-led newspaper. It provided technical training to a dozen youth with the aim of helping sustain the Rafigui newspaper. The project was a resounding success, with follow-up work¬ shops taking place in Burkina Faso in January 2006 and Rafigui now planning its own open-source software-focused workshops for the near future. Now, in July 2007, FMM is finishing a new set of workshops in Kenya with a local organization called Shining Hope for the Community (SHOFCO). Working with 20 youth from Kibera, the seven-week project focuses on training in open-source software and journalism, as well as exploring how Web 2.0 tools can be used within interna¬ tional development. The seeds of the project were planted informally almost a year ago, when SHOFCO received a small grant to pur¬ chase a printer and received an Ubuntu CD in the mail. The printer helped them start the first community-run newsletter in Kibera, while the CD instilled a sense of technical curiosity in the youth. Even today, a year later, we get requests to teach some of the young people how to use Ubuntu. One of the biggest advantages of open- source software and using or creating openly available resources is that such projects easily can be initiated by others. What is surprising is that such projects can end up costing very little and have incredibly fast results. If driven by a motivated organization, the projects often can be sustained long past the initial workshops and have an immense potential to help many people. How to Run a Media Development Project Using Open-Source Software The best piece of advice for starting such a workshop is to be flexible, open-minded and to think critically about the work being done. Such questioning should focus on everything from the general idea of using open-source software to specifics, such as which printer drivers to include on a resource CD. Even the idea of using open-source software was not a trivial one when initially planning workshops in Chad. The specific tasks of newspaper creation—text editing, layout design, photo editing and illustration—are well supported by open-source software packages, such as OpenOffice.org, Scribus, GIMP and Inkscape, but the same may not be true for tasks related to video production or animation. Technical questions are but a small sub¬ set of the ones you will need to answer, and most questions are difficult to fathom or predict. Traveling from a wealthy country to a relatively poor one comes with many challenges surrounding power structures and stereotypes. An illustrative example comes from a former workshop participant, who warned his teachers that students will stay silent, smile and then nod out of respect for the instructor, even if they do not under¬ stand a single word. Teaching anything about computers to new users is not easy, and the difficulty W is compounded when students only have a basic level of literacy or the language of instruction is not their first language. A typical response to such challenges is local¬ ization. Indeed, one of the advantages of using open-source software is the ability to customize user interfaces, either by modifying labels or going the extra mile and re-inventing entire user interfaces. Although many Linux distributions sport numerous languages, incorporating them into lessons is controversial ► at best. The linguistic history of Africa, like any other continent, is complex, and many languages tend to be oral ones, with either no written versions or ones that are barely taught in schools. For example, Swahili is very popular as a spoken language in Kenya, but most people learn to read and write in English in schools in Kibera. Working in Swahili is seen as substandard to English, even if it is conversationally easier. A further problem is that although the operating system may be localized, the likelihood that every package within it is also localized is small, especially if the language being considered is not mainstream. Even with apparently French packages, some help files would say, "To be translated" (in English), causing a great deal of confusion to French- speaking students in Chad. No matter how difficult the software or seemingly ineffective the lesson plans, most students are happy to be given an 16 | October 2007 www.linuxjournal.com [UPFRONT] opportunity to acquire technical training and will, in fact, learn a great deal. We have noticed this regardless of operating system, be it Fedora Core or Windows, or whether we're teaching programming in Java or text editing in OpenOffice.org. The most impor¬ tant aspect of the technical plan should be its simplicity in terms of maintenance and sustainability. Indeed, if you make sure that the computers function long after you are gone and are well stocked with tutorials and books providing instructions on software packages, there's a good chance students will go out of their way to learn what's avail¬ able—just make sure your local partnering organization provides the security to store those resources and gives your workshop participants the ability to access them. In this sense, it is often best to avoid complex software or hardware configura¬ tions. For our work, we often prefer lap¬ tops—they're cheap to transport, have only one plug and use battery power when elec¬ tricity is unavailable (which can be a major problem in Chad and Kenya). We also avoid setting up complex network configurations or user accounts, unless this is something we can teach within the workshops themselves. Conclusion Even if the workshops go well and you end your multiweek experience in bliss, there's often one difficulty creeping about: practical applications. All effort is for naught if, at the end of the weeks, there is no longer-term plan for how the software will be used to improve the organization or start new projects. At the beginning of this article, I mentioned the importance of focusing workshops around a project and how FMM works specifically with organi¬ zations interested in printed media. In this case, we often make a newspaper issue based on workshops and assume the lessons learned will be used to improve the newspaper. In some cases, the bene¬ fits may be more direct, such as helping participants get job offers to move beyond the poverty that has plagued their lives. In many cases, there are no easy answers to questions raised about planning a media development project. Indeed, international development is fraught with ethical issues and cultural challenges. One of the most difficult aspects of such work is that although a project may last several weeks, the end result is the teacher leaving and returning to a lavish lifestyle, while the par¬ ticipants may remain in impoverished areas for years to come, if not their entire lives. Nevertheless, such projects are worthwhile because they bring hope to the communities they focus on, and instructors and organizers often gain unforgettable friendships and experiences. So, the next time you have a vacation or a few weeks free, consider getting involved in such a project. In such cases, open source is more than a software paradigm—it is a ticket out of poverty. — WOJCIECH GRYC For more information on Five Minutes to Midnight or to get involved, please contact Wojciech at wojciech@gmail.com or visit www.a13i.org. Expert Included. When you partner with Silicon Mechanics, you get more than a powerful Intel-based Solution—you get an expert like Forrest. Forrest and the Product Development team never stop thinking about the problems facing today's IT administrators. Power efficiency, compute density, green computing, reliability, and serviceability are just some of the things he makes a priority. That's why Forrest is excited about the new Bladeform 8100 Series Blade Server Platform. With support for multiple Quad-Core Intel® Xeon® Processors 5300 Series, the Bladeform 8100 takes advantage of Intel's proven reliability and energy efficiency. Forrest is very impressed that the Bladeform 8100 provides 90%+ high-efficiency redundant power supplies for operating cost reduction and earth-friendly computing. Up to ten blades, each with 2 removable hard drives, can be installed in the 7U enclosure, allowing 60 total blades in a 42U rack. The Bladeform 8100 is a perfect choice for mission-critical enterprise applications, scale-out, and high performance computing environments. Xeon* inside™ Quad-core. Unmatched. visit us at www.siliconmechanics.com or call us toll free at 866-352-1173 I_I Silicon Mechanics and the Silicon Mechanics logo are registered trademarks of Silicon Mechanics, Inc. Intel, the Intel logo, Xeon, and Xeon Inside are trade¬ marks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. TECH TIPS Spice up man pages, perform keyword searches, and create a logo or mix audio at the command line. » Colorize man Pages Linux man pages are an integral part of Linux folklore. Even seasoned users have to refer to man pages every once in a while. Wouldn't it be much more fun if man pages were a little colorful? If you run a graphical X desktop, it isn't much trouble to add some color to the otherwise very technical and curt manuals. Simply copy the contents of the file called XTerm in the app-defaults directory to your .Xdefaults file. The following replaces the .Xdefaults file entirely: (Ctrl-T), place the cursor at the location bar (Ctrl-L), type your keyword followed by the search term(s)—for example, assuming you added keyword bz for the search at bugzilla.mozilla.org, then typing bz 95849 in the location bar will show you this: https://bugzilla.mozilla.org/show_bug.cgi7icU95849 This is an ideal way to use a search engine that digs through some specific data. —Steve Fernandez $ cp /usr/share/Xll/app-defauUs/XTerm -/.Xdefaults Edit the file and uncomment these lines (or create them if they aren't in your particular file). Change the colors from yellow and red to your favorite colors if yellow and red do not suit you: *VT100*colorULMode: on *VT100*colorllL: yellow !*VT100*italicULMode: on *VT100*underl_ine: off ! Uncomment this to use color for the bold attribute *VT100*colorBDMode: on *VT100*colorBD: red Enjoy the colorful man pages! —Girish Venkatachalam » Keyword Search in Firefox Firefox is a great browser, but you already knew that, right? Firefox's keywords facility can be used for a neat search trick. It is best used for a directed search engine that digs specific data—for example, a Bugzilla search, IMDb search, LXR search or Marcel/wine search (www.wine-searcher.com), and so on. Here are the steps: 1. Go to a site that offers a simple search facility (for example, IMDb, LXR or your local Bugzilla). 2. Place the cursor within the search box. 3. Right-click, and select Add a Keyword for this Search. 4. Give your new search shortcut a name. 5. Give your new search shortcut a short keyword (for example, I use bz for my Bugzilla search and Ixr for LXR). That's it. To try out your new keyword search shortcut, open a new tab » Command-Line Logo You can create a neat logo from the Linux command line: $ convert -size 800x120 xc:white -font Times-Roman -pointsize 100 -fill gray -annotate +20+80 'Linux is cool!' -fill black -annotate +23+83 'Linux is cool!' -trim +repage logo.png And, the following command should display the result: $ qiv logo.png If you want to play with multiple colors and fonts, the following will help: $ convert -list type $ convert -list color —Girish Venkatachalam » Combine Audio Files at the Command Line Say you have two PCM audio files in WAV format. You can concatenate them with the following command (they must both have the same sample rates, encoding, endianness and so on): $ sox filel.wav file2.wav combined.wav If you want to mix two audio tracks, try this: $ soxmix filel.wav file2.wav mix.wav —Girish Venkatachalam Linux Journal pays $100 for tech tips we publish. Send tips and your contact information to techtips@linuxjournal.com. 18 | October 2007 www.linuxjournal.com EmperorLinux ...where Linux & laptops converge Portab Since 1999, EmperorLinux has provided pre-installed Linux laptops to universities, corporations, government labs, and individual Linux enthusiasts. Our laptops range from full- featured ultra-portables to desktop replacements. All systems come with one year of Linux technical support by phone and e-mail, and full manufacturers' warranties apply. Toucan T 61 /T 61 ws ThinkPad T61/T61ws by Lenovo • Up to 15.4" WSXGA+ w/ X@1680xl050 • NVidia Quadra NVS 140M graphics • 1.8-2.4 GHz Core 2 Duo • 512 MB-4 GB RAM • 80-160 GB hard drive • CDRW/DVD or DVD±RW • 5.2-6.0 pounds • 10/100/1000 Mbps ethernet • 802.11a/b/g (54Mbps) WiFi • Starts at $1650 Powerl i EmperorLinux specializes in the installation of Linux on a wide range of the finest laptops made by IBM, Lenovo, Dell, Sony, and Panasonic. We customize your choice of Linux distribution to your laptop and provide support for: ethernet, wireless, X-server, ACPI power management, USB, EVDO, PCMCIA, FireWire, CD/DVD/CDRW, sound, and more. Rhino D830/M6300 Dell Latitude D830/Precision M6300 Up to 17" WUXGAw/ X@1920xl200 NVidia Quadra FX 3500M graphics • 1.8-2.4 GHz Core 2 Duo/Extreme •512 MB-4 GB RAM • 60-160 GB hard drive • DVD±RW or Blu-ray • 6.3-8.6 pounds • 802.11a/b/g (54Mbps) WiFi • ExpressCard/EVDO • Starts at $1365 Unique Ruggedized Panasonic laptops are designed for harsh environments: drops, vibrations, sand, rain, and other extremes. ThinkPad tablet PCs are like other laptops, with an LCD digitizer for pen-based input both as a mouse and with pressure sensitivity for writing and drawing on-screen. Raven X61 Tablet ThinkPad X61 Tablet by Lenovo • 12.1" SXGA+ w/ X@1400xl05 • 1.6 GHz Core 2 Duo • 1-4 GB RAM • 80-120 GB hard drive • 3.8 pounds • Pen/stylus input to screen • Dynamic screen rotation • Handwriting recognition • X61s laptops available • Starts at $2200 www.EmperorLinux.com 1-888-651-6686 Model prices, specifications, and availability may vary. All trademarks are the property of their respective owners. COLUMNS AT THE FORGE Incremental Form Submission REUVEN M.LERNER A creative solution for solving Web service performance bottlenecks, or "ix-nay on the ottleneck-bay”. Computers are amazingly fast. Think about it—we measure raw processor speed by how many instructions it can execute each second, and that number has gotten so large, we round off to the nearest hundred million. Of course, it's often hard to feel that computers are all that speedy, especially when you're sitting around waiting for them to compete a task. Sometimes that wait has to do with complex algorithms that take a while to execute. But in many cases, the problem is a delay further down in the system, which is causing your end-user application to wait for a while. This, I believe, is the major Achilles' heel of the world of Web services—Web-based APIs that are making it increasingly easy to combine (and manipulate) data from multiple sources. Web services may be revolutionizing distributed application development and deployment, but they make it tempting (and too easy, sometimes) to create software whose performance depends on someone else's system. For example, let's assume you are offering a Web service and that your program depends, in turn, on a second Web service. Users of your system might encounter delays at two different points: your Web service (due to computational complexity, a lack of system resources or too many simultaneous requests), Listing 1. pl-words.html Pig Latin translator

Enter the words you wish to translate into Pig Latin:

or the second Web service on which yours depends. Several commercial companies, such as Google, eBay and Amazon, offer Web services to the general public. But, these services lack any sort of uptime or response guarantee and often restrict the number of requests you can make. If you write a Web service that depends on one of these others, a rise in requests to your service might well mean that you temporarily go over your limit with these services. This is particularly true if you allow users to enter one or more inputs at a time. For example, if you're running a Web-based store, you want to let people put multiple items in their shopping baskets. It's easy to imagine a scenario in which each item in the shopping basket requires a call to one or more Web services. If each call takes one second, and if you are allowed to access the Web service only every six seconds, a user trying to buy ten items might end up waiting one minute just to see the final checkout screen. If a brick-and-mortar store were to keep you waiting for one minute, you would be frustrated. If an on-line store were to do the same thing, you probably would just pick up and leave. So, what should you do? Well, you could simply throw up your hands and blame the lower-level service. Or, you could contact the lower-level service and try to negotiate a faster, better deal for yourself. Another option is to try to predict what inputs your users will be handing to you and try to preprocess them, perhaps at night, when fewer users are on your system. I've recently come across this problem myself on some of the sites I've been developing in my consulting work. And, I believe I've found a technique that solves this problem without too much trouble and that demonstrates how Ajax programming techniques not only can add pizazz to a Web site, but make it more functional as well. This month, we take a look at the technique I've developed, which I call (for lack of a better term) incremental form submission. The Problem Before we continue, let's define the problem we are trying to solve. Users visiting our site are presented 20 | October 2007 www.linuxjournal.com Are you Shocked by the high cost of iSCSI & Fibre Channel SAN storage? AoE is the answer! ATA-over-Ethernet = Fast, Reliable, Simple storage. www.coraid.com 1. Fast 10 Gigabit Ethernet Storage without the TCP/IP overhead! 2. Unlimited expandability, at the lowest possible price point!! 3. You want more storage...you just buy more disks - it's that simple!!! Visit us at www.coraid.com EtherDrive® SRxxxx Fast & Flexible RAID appliances with slots for hot swap SATA disks Check out our full line of EtherDrive® Storage and VirtualStorage Appliances and NAS Gateways CORAID www.coraid.com 1.706.548.7200 The Linux Storage People COLUMNS AT THE FORGE Listing 2. pl-words.cgi #!/usr/bin/env ruby # *-ruby-*- require 'cgi' require 'xmlrpc/client' def pl_sentence(sentence) server = XMLRPC::Client.new2('http://127.0.0.1:9000', nil, 240) sentence_array = sentence.split # Send the words, and catch any faults that we find begi n results = server.call("atf.pl_sentence", sentence_array) rescue XMLRPC::FaultException => e exi t # puts "Error:" # puts e.faultCode # puts e.faultString end return results.join(' ') end # Create an instance of CGI cgi = CGI.new("htm!4") # Get the words to translate words = cgi.params['words'] if words.empty? words = '' else words = words[0].downcase end # Send some output to the end user cgi.out { cgi.html { # Produce a header cgi.head { cgi.title { "Your Pig Latin translation" } } + # Produce a body cgi.body { cgi.hi { "Pig Latin translation results" } + cgi.p { "Original sentence: ’#{words}'" } + cgi.p { "Translated sentence: '#{pl_sentence(words)}'" } } } } with an HTML form. The form contains a textarea widget, into which users can enter one or more words. When a user clicks on the submit button, a server-side program takes the contents of the textarea and sends it to a Web service that turns each word into its Pig Latin equivalent. The server-side program retrieves the results from the Web service and displays the Pig Latin on the user's screen as HTML. It goes without saying that this example is some¬ what contrived; although it might be nice to have a Web service that handles translations into Pig Latin, it takes so little time to do that translation (really, a simple text transformation), that storing or caching this information would be foolish. That said, this example is meant to provide food for thought, rather than a production-ready piece of software. Let's start with our HTML file, shown in Listing 1. It contains a short HTML form with a textarea widget (named words) and a submit button. Clicking on that submit button submits the form's contents to a CGI program written in Ruby, named pl-word.cgi (Listing 2). There are two sections to Listing 2. In the first part of the program, we define a method, pLsentence, that takes a sentence (that is, a string), turns it into an array of strings (with each word in one string), and then passes that array to our Web service (via XML-RPC). The second half of the program takes the input from our POST request, passes it to the pLsentence routine, and then uses the output from pLsentence to create a bit of nicely formatted (if Spartan) output for the user. The key to making all this work is shown in Listing 3, which provides the code for our XML-RPC server. We begin by reading from a simple cache of English words and their Pig Latin equivalents. Again, it seems silly to store things in this way, when it's much faster simply to write the code that handles the Pig Latin rules. If you imagine that each translation takes several seconds, you can see how things could pile up quickly. There are several things to notice in this program. One of the first is the use of an on-disk cache to store recently processed inputs. (Please don't try to emulate the simple and foolish way in which I implemented this; I ignored locking and permission issues.) The cache itself is a simple text file containing name-value pairs. Before computing the Pig Latin translation of each item, the Web service consults the cache. If the word is in the cache, the service grabs that value and almost immediately returns the translated value. If the word isn't in the cache, it translates the English into Pig Latin, storing the values for the next time around. Again, this ensures that we have to work hard (that is, translate the word into Pig Latin) only if it fails to appear in the cache. If you've never programmed in Ruby before, you might be put off a bit by this line: 22 | October 2007 www.linuxjournal.com SMART, CURIOUS, INQUISITIVE, MOTIVATED. Daring to take on the challenges that others won't. This is a White Oak employee on any given day. Tackling ambitious objectives that push your limits. Working with passionate people who inspire greatness. Finding ingenious solutions to the toughest problems. On the job or off, White Oak people keep their adrenaline flowing! Our benefits are as exceptional as the people we hire. If you have impressive architecture and development experience in one or more of the following areas, show us what you can do. Send an email to: bruce@woti.com. • Web and Database Technologies, AJAX • C++, Python, Perl, Linux • User Interface Development • Parser & Compiler Design • Parallel Algorithms and Scalable Data Structures • TCP/IP Programming frWhiteQak ^"TECHNOLOGIES, INC? Work hard, think hard, play hard! Positions located in the Washington, D.C. area. Relocation packages available for qualified candidates. www.woti.com US Citizens only. White Oak Technologies, Inc. is an Equal Employment Opportunity Employer. COLUMNS AT THE FORGE words.map {|word| word.to_s}.each do |word| This tells Ruby that it should take the array named words and turn each of its elements into a string. (If the element already is a string, it is unaffected.) We then iterate over each string (word) in the array, assigning the local variable word to each element in sequence. With Listings 1, 2 and 3 in place, you should be able to translate sentences from English into Pig Latin without too much difficulty. You enter the English words into the HTML form, the server-side program calls the Web service, and the Web service takes care of things quickly. Improving Performance Now we come to the hard, or interesting, part of this project. If you can imagine that each Pig Latin translation takes ten seconds to execute, but less than one second to retrieve from the cache, you would want the cache to be used as much as possible. Moreover, given how long each word lookup takes, users will need a great deal of patience to deal with it. The solution? Use Prototype, a popular JavaScript framework. Its AjaxUpdater will submit the contents of the textarea widget to a URL of your choice automati¬ cally—in this case, the same one that is used for POST—in the background, each time the text area is changed. Then, each word is translated while the user Listing 3. pl-server.rb #! /usr/bin/ruby require 'rubygems' require ' net/http' require 'rexml/document' require ' xmlrpc/server' # - # Load the translation cache # - dictionary = { } puts "Loading cached translations" translation_file = 'translations.txt' if FileTest.exists?(translation_file) File.open(translation_file, "r").each do |line| (english, piglatin) = line.chomp.sp1it(' = ') dictionary[english] = piglatin puts "'#{eng1ish}' => '#{piglatin}'" end else File.open(translation_file, ’w') do jlinej end end # - # XML-RPC # - # Start an HTTP server on port 9000, to listen for clients server = XMLRPC::Server.new(port=9000, host='127.0.0.1') server.add_handler(name="atf.pl_sentence", signature^['array', 'array']) do |words| words.map {|word| word.to_s}.each do |word| # Have we already seen this word? Don't bother to # translate it if dictionary.has_key?(word) puts "Grabbing translation of '#{word}' from the dictionary" output << dictionary[word] next end # If it's not in the cache, then go for it. piglatin = ' ' if word =~ / A [aeiou]/ piglatin << word piglatin << 'way' else piglatin = word[1..-1] piglatin << word[0] piglatin << 'ay' end puts "Translated '#{word}' => '#{piglatin}'" # Cache it puts "Trying to cache..." dictionary[word] = piglatin File.open(translation_file, 'a') {|f| f.puts "#{word}=#{piglatin}"} output << piglatin end output output = [ ] 24 | October 2007 www.linuxjournal.com Web services may be revolutionizing distributed application development and deployment, but they make it tempting (and too easy sometimes) to create software whose performance depends on someone elses system. is filling out the text form, dramatically reducing the time needed to translate. In other words, I'm betting it will take enough time for users to enter the entire sentence, that I can collect and translate most or all of the translated words while they're typing. Also, because I know that the Web service is caching results, I can pass the contents of the entire textarea every few seconds, knowing that retrieving items from the cache is extremely rapid. The key to this functionality is the use of the Form.Element.Observer object in JavaScript. This object allows us to monitor any form element over time, submitting the form's contents to an arbitrary URL when the form element changes. We will use this, along with our knowledge that the Pig Latin server (pl-server.rb) caches words it has already translated, to submit the form every few seconds, even before the user clicks the submit button. We do this by adding an id attribute, whose value is words, to our textarea, and also by adding the following JavaScript code: new Form.Element.Observer($("words"), 3, translateFunction); In other words, we will check the words in textarea for changes every three seconds. If something has changed, the browser invokes the method translateFunction. This function is defined as follows: function translateFunction() { var myAjax = new Ajax.Request( '/pl-words.cgi ' , { parameters: Form.seriatize('form') }): } In other words, translateFunction creates a new Ajax request in the background, submitting the contents of the form to the URL/pl-words.cgi—the same program to which the form will be sub¬ mitted at the end of the process. But, for our incremental submissions, we care more about the side effects (that is, the cached translations) than the resulting HTML. So, we ignore the output from pl-words.cgi. Because of how we built our server-side programs, they don't need to change at all in order for this Ajax-style addition to take effect. All we need to do is modify the HTML file, adding a few lines of JavaScript. Now, of course, this doesn't change the amount of time it takes to translate each word or even an entire sentence. But, that's not the Windows Price, Quality, & Service 1U Supermicro 6015B-TV: (Customized System) 2x Xeon® Quad-Core E5335 2GHz CPU, 4GB 667mhz FB-DIMM, 2x 250GB HD, Slim DVD.. 2U Supermicro 6025B-TR+B: (Like above spec. 2x Xeon® E5335 CPU, 4GB RAM, 2x 250GB HD w/ Redundant Power, 8 hotswap bays .... *Free gadget w/ purchase! 5ft fedora® soiaris Customizable system solutions since 1989 Tel: 1-800-875-8590 Fax:408-736-4151 KING STAR COMPUTER 1259 Reamwood Ave .Sunnyvale, CA 94089 www.kingstarusa.com Ema il : sa l es@ki n gstaru£ a.com Intel©, Intel© Xeon , Intel Inside©, Intel© Itanium© and the Intel Inside© logo are trademarks or registered trademarks of Intel Corporation of its subsidiaries in the United States and other countries. Prices and availability subject to change without notice. Not responsible for typographical errors. COLUMNS AT THE FORGE Listing 4. Ajax Version of pl-words.html Pig Latin translator

Enter the words you wish to translate into Pig Latin:

point. Rather, what we're doing is taking advantage of the fact that many people tend to type slowly and that they'll take their time entering words into a textarea widget. If users type quickly, or enter a very short sentence, we haven't really lost anything at all. It'll take a long time to translate those people's sentences, and they'll just have to wait it out. If people change their minds a great deal, it's possible we'll end up with all sorts of cached, translated words that are never going to be used again. But, given that the cache is shared across all users, it seems like a relatively small risk to take. There are some things to consider if you're thinking of going this route—that is, combining an incremental form submission with a cache. First, notice we are iter¬ ating over each word in the textarea. This means there's the potential for someone to launch a denial-of-service attack against your server, simply by entering ridiculously long text strings into your textarea widget. One way to prevent this is to limit the number of words you check from any given textarea widget. You can, of course, limit the number of words you're willing to translate from the incremental submission, rather than from the complete and final submission. Another item to remember is that you should not expose your inner APIs. APIs are for external use; the moment people know your internal data structures and methods, they might use them against you. These examples didn't include any cleaning or testing of the data that was passed to the server; in a real-world case, you probably would want to do that before simply passing it along to another program. Finally, if your site becomes popular, you might need more than one server to handle Web services. That's fine, and it's even a good idea. But, how many servers should you get, and how should they store their data? One possibility, and something that I expect to write about in the coming months, is Amazon's EC2 (Electric Computing Cloud) technology, which allows you to launch an almost limitless number of Web servers quickly and for a reasonable price. Combining EC2 with this sort of caching Web service might work well, especially if you have a good method for sharing dynamic data among the servers. Conclusion Web services are a wonderful way for servers to share data. But, when a Web service becomes a bottleneck, and when we lack control over the size of the bottleneck, we must try to find creative solutions. This month, we looked at something that I call incremental posting, designed to spread the burden over time, as a user is typing. Even if this solution isn't quite right for you, perhaps you'll be inspired in some way to incorporate this, or other Ajax techniques, into your own sites. ■ Reuven M. Lerner, a longtime Web/database developer and consultant, is a PhD candidate in learning sciences at Northwestern University, studying on-line learning communities. He recently returned (with his wife and three children) to their home in Modi’in. Israel, after four years in the Chicago area. Resources Programs this month were written in Ruby, a popular general-purpose programming language. You can read more about Ruby, and download or browse through the documentation, at ruby-lang.org. If you want to learn more about the Prototype library for JavaScript, go to www.prototypejs.org You can learn about Ajax programming techniques from many sites. One of my favorite books on the subject is Ajax Design Patterns by Michael Mahemoff, published by O'Reilly. I also have found it useful and interesting to keep up on the (seemingly limitless) amount of JavaScript and Ajax news at www.ajaxian.com. 26 | October 2007 www.linuxjournal.com PGI Unified Binary" Now, PGrcompilers can generate a single PGI Unified Binary executable fully optimized for both Intel EM64T and AMD64 processors, delivering all the benefits of a single x64 platform while enabling you to leverage the latest innovations from both Intel and AMD. PGI Fortran, C, and C++ compilers deliver world-class performance and a uniform development environment across Linux and Windows as part of an integrated suite of multi-core capable software devel¬ opment tools. Visit www.pgroup.com to see why the leading independent software vendors in structural analysis, computational chemistry, computational fluid dynamics and automotive crash testing choose PGI compilers and tools to build and optimize their 64-bit applications. ‘ ^ The Portland Group” www.pgroup.com ++ 01 (503) 682-2806 The Portland Group, Inc. is an STMicroelectronics company. PGI, The Portland Group, PGI Unified Binary are trademarks or registered trademarks of STMicroelectronics. Other brands and names are the property of their respective owners. COLUMNS COOKING WITH LINUX Your Voice, Forever Etched in Electrons marcel gagne You want to record a simple voice message on your Linux system. It seems like an easy enough thing—until you need a certain audio format, complex edits, special effects and other audio tweaks. However, it’s still pretty easy. KMix File Settings Help Output input ^ . 0 o= LLI o That must be the 30th time I've heard you repeat that phrase, Frangois. What are you doing? Trying to record a new voice message for our Asterisk Linux-based answering machine? But you keep repeating yourself. Don't you like any of the recordings you've made so far? Quoi? None of them have worked? Ah, here is the problem...the micro¬ phone isn't on. Wait! I see a second, similar, problem. Your mixer gain is set all the way down. Now try it. Much better, non? Finish this later, Frangois, I can see that our guests are already arriving and we must be ready. Look sharp. Welcome, mes amis , to Chez Marcel , where the best in Linux and open-source software is paired with superb wine from around the globe. Makes yourselves comfort¬ able, and I will send my faithful waiter to fetch tonight's wine selection. Frangois, we have a few bottles left of that Niagara Region 1998 Reif Estates Vidal Ice wine. Please fetch them from the cellar. On tonight's menu, mes amis , we will examine sound recording tools for your Linux system. Recording audio is actually pretty simple, though you must remember to turn up the microphone on your desktop's mixer applet (Figure 1). Some systems, notebook computers for instance, have multiple inputs. In addition to the internal, built-in micro¬ phone, you may also have a jack to plug in a headset, which has its own microphone. Make sure you select the appropriate source. Getting a sound sample is easy enough and doesn't require fancy software. Command-line tools that are very likely already included in your system will do the job nicely. For instance, look for a command called arecord, which comes with a companion program called aplay. Simply put, arecord is a sound recorder for the ALSA (Advanced Linux Sound Architecture) subsys¬ tem. Here it is in its simplest form: _ n x Switches s £> = E O HDA Intel Figure 1. Remember to set your microphone gain before you record. arecord -d 10 myrecording.wav The result is a WAV format file named myrecording.wav that is ten seconds in length. The WAV format is the default, so if I hadn't given my file an extention, the result still would have been a WAV file. Other file formats (au, voc or raw) can be specified on the command line with the -f option. Plenty of options let you change the sample rate, number of channels and so on. Run the command with a -h, or check the man page for a list of available options. Here's another example: arecord -d 15 -f cd secondrecording.wav The really interesting option here is -f cd, a shortcut for -f S16_LE -c2 -r44100, which effectively means 16- bit, little endian, 2-channel sound and a 44,100Hz sample rate. Then, if you want, you can play that clip using the aplay command. There are, of course, some limitations with such a simple program. For instance, what if you want another audio format? Or, what if you want to do some kind of special effect using that clip? This is where another great little command-line program comes into play. It's called sox, the SOund exchange universal sound sample translator. Let's say you want to convert an MP3 file to OGG; sox makes it easy: sox audiofile.mp3 audiofile.ogg Simple, non? The sox program also has a set of effects you can apply to your sound clip. For instance, let's do a two-second fade in for a voice clip: sox voicefile.ogg newfile.ogg fade 2 As you can see, it's not difficult. To get a handle on the various effects, type sox --help-effects=all. There are, of course, graphical tools for recording sound. For instance, the KDE desktop's multimedia suite includes a program called KRec (Figure 2), a very capable sound recorder that takes advantage of KDE's aRts sound system. Because of this aRts integration, make sure your aRts daemon is set to use full-duplex mode—you can enable this in the KDE Control Panel, kcontrol, under the Multimedia section. To record a sound with KRec, start by clicking the New 28 | October 2007 www.linuxjournal.com Polywell's Ultimate Linux Systems More Choices, Excellent Support Service, Great Value! 1U Value Servers Starts at $399, Up to 8GB RAM i?1 1 j nux inijiiance starts at $299 $3S9 1U-485AX Sempron 3000+, 1GB DDR2,80GB HD (For Volume Purchase Only) $499 1U-485AX Athlon64 3500+, 2GB DDR2, 80GB HD (For Volume Purchase Only) $699 1U-690GA Athlon64 X2 Dual-Core, GE DDR2, 2x160GB HD (Dual LAN +$45) 1U-690GA Athlon64 X2 Dual-Core, GE DDR2, 2x160GB HD (Dual GigaLAN +$45) Poly 6100Nx Athlon64 X2, 512MB RAM Nvidia 6100 Graphics, 80G, CD-ROM $299 Poly 7050A Athlon64 X2 4200+, 1GB DDR2 Nvidia 7050 Graphics, 500G, DVD-RW $499 (custom config. available) (OEM /ODM Service Available) $1,799 1U4S-690A Athlon64 X2 6000+, 8GB DDR2 2TB 4x500GB HD, Dual Gigabit LAN $2,999 1U-2500A16 2 x Opteron 2212,16GB ECC DDR2, 1TB 4x250GB HD, Dual Gigabit LAN $4,999 1U-2500A16 2 x Opteron 2216,32GB ECC DDR2, 2TB 4x500GB HD, Dual Gigabit LAN (Option: 64GB+4TB HD) $10,999 1U-8415SS32 4 x Opteron 8212,64GB ECC DDR2, 2x73GB SAS HD, 3x Gigabit LAN (Option: 128GB RAM) $999 Netdisk 4000 2.0TB 4x500G $1,499 Netdisk 6000A 3TB 6x500G (2U) $2,999 Netdisk 6000B STB 6x1000G (2U) 1U Twin Servers, 1U 8-Way Quad Opteron $4,999 lU-Twin 2x2 Dual-Core Processors, 2 x 8GB ECC DDR2, 2 x Dual 250GB HD, 2 x Dual Gigabit LAN $5,999 1U-8415A 4 x Opteron 8212 Dual-Core, 16GB ECC DDR2, 1.5TB 3x500GB HD, Dual Gigabit LAN $7,999 1U-8415SS 4 x Opteron 8212 Dual-Core, 32GB ECC DDR2, 2x73GB 15K RPM SAS HD, 3x Gigabit LAN High-Density Multi-Processor Servers 1U to 5U, Quad to Eight Processors, Up to 128G RAM 6TB 2U Storage Server 2012SC-2055A 4TB 8x500G, Opteron $2,999 6TB 12X500G, Opteron $3,999 $11,999 2U-8425SS 4 x Opteron 8212,64GB ECC DDR2, 4TB 8x500GB RAID-5 Storage, 3 x Gigabit LAN $32,999 2U-8425SS 4 x Opteron 8212,128GB ECC DDR2, 2x73GB 15K RPM SAS HD, 3x Gigabit LAN $16,500 5U-8850T5U 8 x Opteron 8212,64GB RAM, 2TB 4x500GB HD, 3 x Gigabit LAN $36,999 5U-8850T5U 8 x Opteron 8212,128GB RAM, 4TB 8x500GB HD, 3 x Gigabit LAN Blade Servers -10 Dual or Quad Processors Blades 24TB 4U Storage Server 4024SS 4U 24-Bay 12TB 24x500G, Opteron $7,500 24TB 24x1000G, Opteron $12,999 $13,999 8U 10 x 2055A Blades 10 x (Dual Opteron 2210,4GB RAM, 80G HD) $24,999 8U 10 x 2500A16 Blades 10 x (Dual Opteron 2212, 16GI RAM, 80G HD) $46,999 8U 10 X8450A Blades 10 x (Quad Opteron 8212,16GB RAM, 80G HD) $69,999 8U 10 x 8450A Blades 10 x (Quad Opteron 2214, 32GB RAM, 80G HD) AMD Dual-Core technology enables one platform to meet the needs of multi-tasking and multi-threaded environments; provides platform longevity 20 Years of Customer Satisfaction 5-Year Warranty, Industry's Longest First Class Customer Service Polywell OEM Services, Your Virtual Manufacturer Prototype Development with Linux/FreeBSD Support Small Scale to Mass Production Manufacturing Fulfillment, Shipping and RMA Repairs Polywell Computers, Inc 1461 San Mateo Ave. South San Francisco, CA 94080 650.583.7222 Fax: 650.583.1974 Opteron, Sempron and ATHLON are trademarks of Advanced Micro Devices, Inc.. Quadro, nForce and Nvidia are trademarks of NVIDIA Corporation. All other brands, names are trademarks of their respective companies. POLYWELL AMDH AMDC1 Opteron - Athlon X2 COLUMNS COOKING WITH LINUX Getting a sound sample is easy enough and doesn’t require fancy software. Command-line tools that are very likely already included in your system will do the job nicely. Figure 2. KDE’s KRec tool is surprisingly complex under the surface, but it makes a great tool for simple audio recordings. icon on the top left (or click File on the menu bar and select New). Then, click the red Record button and begin speaking into your microphone. Recording and playback levels are displayed, as is positional informa¬ tion on the raw data being recorded. At the bottom, there are a series of dials that let you adjust the aRts compressor settings. When you have finished recording, press the Stop button, rewind, then press Play to listen to your clip. When you are ready to save it, you can choose to keep the raw data or export it to a more familiar format, such as WAV, OGG or MP3. Click File on the menu bar and select Export. For GNOME desktop users, we have the GNOME sound recorder (Figure 3), which is available from the Multimedia menu. Using that program, you can select your input source (internal or external microphone and so forth), and the audio file format from the Record as drop¬ down box. To start recording, click the red Record button and start talking, singing or reciting Shakespearean poet¬ ry—whatever turns you on. When you are done, click the Stop button (the gray square at the end of the icon bar). Then, click the Save button. These are all easy ways to record sound, but editing is Figure 3. The GNOME desktop provides an easy-to-use tool for simple recordings. limited. You have only so much control over recording quality, and anything other than the simplest of edits can be difficult or impossible. That's where a program like Audacity comes into play. Audacity is a wonderful, easy-to-use, audio editing program. With Audacity, you can record audio from a vari¬ ety of sources, including a microphone—podcasts, anyone? You also can use it to convert audio files into other audio formats. Take your old records or tapes, clean up the noise, and convert them to digital audio so you can burn them to CD. Edit, cut, copy, mix, add special effects and splice sound sources to create new sounds. Audacity is a multitrack real-time audio editing system that can handle 16-, 24- and 32-bit samples. Audacity is also just plain fun. You can get a copy of Audacity from your favorite Linux distribution's repositories (or install disks), or you can visit audacity.sourceforge.net for the latest source. In the following examples, I am using version 1.3.3 beta. Audacity starts with a blank slate by default (Figure 4). Along the top of Audacity's main window, you will find a pretty standard menu bar with access to various categories of tools in Audacity's toolbox. Directly below the menu bar and toward the center, a number of buttons reflect Audacity's audio editing nature. These buttons are Pause, Play, Stop, Skip to Start, Skip to End and Record. I mention these first because they are so familiar. Figure 4. Audacity presents a blank slate for your creative urges. To the right of those buttons is a compact toolbox with six small icons representing some common tools used in Audacity. The vertical bar icon, which looks like a capital I, is the Selection tool, and it is selected by default. Now, let's record something. Make sure your microphone is plugged in, and then click the Record button to start. Be creative. Sing a short tune, recite a line or two of poetry, or just speak whatever nonsense pops into your head. As you record, keep your eye on the microphone icon near the top on the far right. If you pause your mouse cursor over it, the tooltip reads, "Input level monitor - click to monitor input". When using a stereo input source, you'll see both the left and right channel levels being displayed, as shown in Figure 5. Of course, if you are using a single-channel | October 2007 www.linuxjournal.com 30 MAXIMIZE PROCESSING PERFORMANCE AND MAXIMIZE RESPONSIVENESS. SDR-I476T STARTING AT $1,799 • Quad-Core Intel® Xeon® 5310 processor • Intel® 5000P Chipset • 1024MB FB-DIMM ECC Memory • 750GB 7200RPM 16MB Drive SDR-I2500T STARTING AT $1,999 • Quad-Core Intel® Xeon® 5310 processor • Intel® 5000P Chipset • 1024MB FB-DIMM ECC Memory • 750GB 7200RPM 16MB Drive i 1 -^ 00 - 57^7931 servers D!RECT Servers Direct 1-877-727-7887 www.ServersDirect.com SERVERS DIRECT CAN HELP YOU CONFIGURE YOUR NEXT HIGH PERFORMANCE SERVER SYSTEM-CONTACT US TODAY! Our flexible on-line products configurator allows you to source a custom solution. Or call—our product experts are standing by to help you assemble systems that reguire a little extra. Servers Direct—your direct source for scalable, cost effective server solutions. Quad-core Unmatched © 2007 Intel Corporation. All rights reserved. Intel, the Intel logo, Intel. Leap ahead., the Intel. Leap ahead, logo, Xeon, and Xeon Inside are trademarks of Intel Corporation in the U.S. and other countries. *Other names and brands may be claimed as the property of others. COLUMNS COOKING WITH LINUX Change your pitch (without changing tempo), change the tempo, equalize soft and loud portions of your audio, add tremolo, remove noise and more. microphone, you'll see only the right channel. Fi‘£l An, ily .' 1 H*‘lp TT-fr p 9 * 1 dj; _ie£ ..1 IT MHt -■ P P ft £ Figure 5. While you are recording, keep your eye on the input level meter on the top right. As you record, you will see the appearance of an audio track with details about the quality of the recording— whether it's a mono or stereo recording and so on. When you are finished recording, click the Stop button. The full audio track remains with timing marks above (Figure 6). Figure 6. With a voice sample recorded. Audacity now displays one audio track. As you can see from the preceding sample, I recorded a little more than six seconds of speech. To listen to the recorded track, click the Play button (Figure 7). folder that will house your project. Give your project a name (I'll call mine justplaying), and then click Save. When you save a project, everything having to do with your project is saved, as it is at that moment. The only thing to remember is that Audacity project files (with an .aup extension) cannot be opened by other packages. The AUP file is accompanied by another folder of the same name, but with a _data extension. Now that your project is safe and sound, let's do some edits on that file. Basic audio editing consists of identifying a section of track, selecting that section and performing some action on that section. Notice the first second or so of my recorded sample in the closeup in Figure 8. Yes, it's the dreaded dead air, the mini-uhm we tend to sneak in at the beginning of these things. I got lucky with that small pause, but it can often be a lot worse. Click the beginning of the sample at the zero mark, and drag the mouse pointer to select that pause. Figure 8. The first second of dead air in my recording is selected in preparation for trimming. prt tin yew jraeki fienwnB' Effect JJJJ 4 * " * ' Q Figure 7. When playing back the sound clip, look at the meter directly to the right of the Record button. At the bottom of the screen, there are additional details on the recorded track, the project audio rate (more on that shortly), as well as positional information. Now that you have a sound clip to work with, this is a good time to save your work, and a good time for Frangois to refill everyone's glass. While he does so, let me tell you about exactly what you are saving. At this stage, you do not have a finished product, but a work in progress. Audacity calls these projects. To save your pro¬ ject, click File on the menu bar and select Save Project As. A file navigation dialog appears where you can select the After you've selected the section of audio that contains the dead air, click the Play button to make sure that you haven't selected a portion of your speech. If necessary, adjust the selected area by positioning your mouse cursor over the beginning or end of the selected area and drag¬ ging to the left or right. The cursor changes to a hand with a pointing finger. If you are satisfied with your selection, click Edit on the menu bar and select Cut (you also can press the Delete key). Now, click the Play button again to lis¬ ten to your file without that little bit of dead air. If you make a mistake, you can undo the changes by pressing Ctrl-Z. With Audacity, you can let your creativity run wild with tons of included effects. Let's say, for example, you want to fade out the last few seconds of your recording. Select that section of the audio track, play it first to confirm you have what you want, and then click Effect and select Fade Out. Perhaps you need to emphasize a few words. Again, select the section of audio you want, click Effect, and select Amplify. In the dialog that appears (Figure 9), amplify your selection by using the slider for a decibel increase. For finer | October 2007 www.linuxjournal.com 32 Figure 9. Despite the name. Amplify, this dialog can be used to decrease the volume as well. Figure 10. The repeat effect identifies the length of the segment and then asks how many times you want that segment repeated. control, simply type the number into the Amplification (dB) field. I should point out that despite the name, Amplify, you can enter a negative amplification to reduce the volume. Click the Preview button to sample the effect before you click OK. Sometimes, repetition is the best way to get your point across. Make your selection, click Effects on the menu bar, and select Repeat. The default is to repeat the selected audio ten times, but you can override that in the dialog (Figure 10). This repeat effect can be a lot of fun if you select a very short seg¬ ment (or a single word) and set it to repeat for several beats. I highly recommend that you spend some time playing with the effects—-there are plenty of them. Aside from being a great way to waste time, you'll be impressed with the arsenal of effects at your disposal. Change your pitch (without changing tempo), change the tempo, equalize soft and loud portions of your audio, add tremolo, remove noise and more. The latest version of Audacity also has tons of additional plugins (under the Effect menu) to keep you entertained. Okay, I want to cover one last effect, which I'll call "Nostalgia Time at Chez Marcel ". One of my favorite effects is something those of us who can still remember vinyl albums will appreciate. On the occasional album, there were sections of a recording where you could play the sound backward to reveal a secret message. Granted, some of these so-called hidden messages were imagined, and Cool Clip.mp 3 ■ EjrawiP far trihn-r fatour* MP3 Hlf^ jpOnn.,. playing your album backward did nothing but add wear and tear to your needle, but others really were there. Well, you can create your own hidden mes¬ sage by using the Reverse effect. So, where do you go from here? You've created some cool sound samples, played with them, cut and trimmed them, amplified here, reverbed there, changed pitch and tempo, and otherwise created something totally new from what started out as a simple voice clip. What else is there? One option is to create your own podcasts, and I mention this mostly to bring up this point. When you are finished with your masterpiece, you want to save that file in a format that your listeners can use—that might be MP3, OGG, WAV or something else. Remember, up to this point, you are dealing with Audacity project files only. Click File on the menu bar and select Export. The Export File dialog appears (Figure 11) from which you can select a filename, location and type. Mon Dieu! That clock on the wall cannot be right! Can it already be closing time? Ah, mes amis , I fear it may indeed be correct. Perhaps we can convince Frangois to refill everyone's glass one final time before we must all head out into the open night. Raise your glasses, mes amis, and let us all drink to one another's health. A votre sante! Bon appetit! ■ tawel Figure 11. You can export (or save) your audio creation in a variety of formats, including MP3. Marcel Gagne is an award-winning writer living in Waterloo, Ontario. He is the author of the all-new A loving to Free Software, his sixth book from Addison- Wesley. He also makes regular television appearances as Call for Help’s Linux guy. Marcel is also a pilot, a past Top-40 disc jockey, writes science fiction and fantasy, and folds a mean Origami T-Rex. He can be reached via e-mail at mggagne@salmar.com. You can discover lots of other things (including great Wine links) from his Web site at www.marcelgagne.com. Resources Audacity: audacity.sourceforge.net GNOME: www.gnome.org KDE: www.kde.org Marcel's Web Site: www.marcelgagne.com The WFTL-LUG, Marcel's Online Linux User Group: www.marcelgagne.com/wftllugform.html www.linuxjournal.com October 2007 | 33 COLUMNS WORK THE SHELL Yahtzee as a Shell Script? When Will It End? dave taylor Many a tear has to fall but it’s all in the game. We seem to spend a lot of time talking about games and how to program them as shell scripts, don't we? From Blackjack to Baccarat , we're in danger of having to rename this column "game programming in the shell". But, that'd be crazy; who in the heck would write multiple games as shell scripts? So, this month, I thought it would be fun to look at a dice game and see how the basic set of playing card functions we've written previously compare to the necessary functions to play a dice game. Yahtzee was first introduced by Hasbro in 1956 as Yacht (having been invented by a wealthy couple on their fancy boat) and has been one of its best-selling titles since, spawning many variants, including hand¬ held electronic games and more. At its heart though, it's basically five-card draw poker played with dice. The wrinkle is that there are a set number of possible hands you can roll, and you attempt to achieve them all to maximize your score. For example, roll a 3 4 4 4 5, and you might well pick up the 3 and the 5, hoping for either "your fours" (which you can get only once and want to choose when you have the maximum number of fours showing), or if you get five of a kind, a "Yahtzee", which is a big-points bonus but obviously difficult to achieve. Like five-card draw, you can pick up zero to five dice and reroll them, but unlike five-card draw, you can do this twice on your turn, not once. So, perhaps the 3 4 4 4 5 rerolls as a 1 4 4 4 4. The second roll would then be to reroll the one and hope for another four. Either way, it's a good roll (unless you've already marked your fours). Modeling It All Dice are quite easy to create in a script—so easy it reveals how straightforward a script like liar's dice would be to write: function rollDie() { dieroll=$(( ( $ RANDOM % 6 ) + 1 )) } If it's this easy to roll a die, though (dice, by the way, is plural of die), it'd be darn easy to write a quick Dungeons and Dragons dice roller too, as shown: function rollDie() { sides=${l:-6} echo "testing with a $sides-sided die...." dieroll=$(( ( $RANDOM % $sides ) + 1 )) } All you need to do is call rollDie with the number of sides you want on the dice it needs to roll. Using a 20-sided die? Try rollDie 20 to see what rolls. This also can quickly and easily be converted into a command-line function, so you could be a real D&D nerd by having a laptop adjacent and typing in roll 20 every time you're actually supposed to roll the die. But back to Yahtzee, yes? The easy part of modeling the game is the dice rolls. We need to have five dice, and that easily can be done with an array: rollDie ; dice[0]=$dieroll rollDie ; dice [1]=$dieroll rollDie ; dice [2]=$dieroll rollDie ; dice[3]=$dieroll rollDie ; dice[4]=$dieroll There, that's your first roll of the five dice. Displaying the results also is easy: echo "You rolled 8 ${dice[0]} ${dice[l]} ${dice[2]} ${dice[3]} ${dice[4]} Note carefully where I do and don't need to use the curly braces to get the array to work properly in the shell. Try this to see how it differs: echo "You rolled " $dice [0] $dice[1] Quite different results, as you can see. (And, as usual with shell programming, there's no useful warnings or error messages to clue you in to what might be wrong.) Rerolling Specific Dice Rolling the dice to get an initial hand is pretty straightforward, so let's take the next step and write the code to let you reroll any or all of the five dice twice to get your final hand. There are a number of ways to ask for this sort of input, but to make it a bit chatty, let's simply present each 34 | October 2007 www.linuxjournal.com Continuous Data Protection The Future of Data Centers Can your backup software do this? RISoft CDP Server Acronis® True Image EMC Retrospect ® Daily Backups Hourly Backups Open File Backups Bare-Metal Restore Continuous Data Protection Restore Linux LVM Restore Linux Software RAID Easy To Use Web Interface Manage Thousands of Servers Control Panel Integration Not Supported Not Supported Not Supported Not Supported Not Supported Not Supported Not Supported Not Supported Not Supported Not Supported 1/ Not Supported Not Supported Not Supported Not Supported Not Supported Not Supported $80-$100/server $699/server You Can't Afford It Data Centers serious a bout uptime and performance use RISoft. For more information visit: www.r1soft.com or call us at 800-956-6198 Copyright 2007 Righteous Software Inc All Rights Reserved. R1 Soft is a trademark of Righteous Software Inc. Other names may be trademarks of their respective owners. COLUMNS WORK THE SHELL Listing 1. yahtzee.sh #!/bin/sh function rollDie() { sides=${l:-6} dieroll=$(( ( $RANDOM % Ssides ) + 1 )) } rollDie ; dice[1]=$dieroll rollDie ; dice [2]=$dieroll rollDie ; dice[3]=$dieroll rollDie ; dice[4]=$dieroll rollDie ; dice[5]=$dieroll echo -n "You rolled [${dice[1]}] , [${dice [2]}] , [${dice [3]}] , echo " [${dice [4]} ] and [${dice [5]} ]" echo -n "Reroll which dice? " read answer for reroll in $answer do if [ $reroll -It 1 -o $reroll -gt 5 ] ; then echo "Invalid entry: $reroll. Please enter 1-5" else rollDie dice[$re roll]=$dieroll fi done do if [ $reroll -It 1 -o $reroll -gt 5 ] ; then echo "Invalid entry: $reroll. Please enter 1-5" else echo "Requested: $reroll" fi done Now, of course, it's time for some actual logic here, not merely a rudimentary test. I've simplified things just a wee bit by using array indices 1-5 rather than 0-4, sacri¬ ficing the slot of entry 0 so that it's easier to work with the values. This means if you ask to reroll die 4, for example, it's just a reassignment of dice[4], Here's the new, improved for loop: for reroll in $answer do if [ $reroll -It 1 -o $reroll -gt 5 ] ; then echo "Invalid entry: $reroll. Please enter 1-5" else rollDie dice [$re roll]=$dieroll fi done You can see that it's quite simple, and if we're not afraid of the code stretching out a bit, we simply can copy and paste some of it to show our before and after rolls: echo -n "Your new roll: [${dice[1]}] , " echo -n " [${dice [2]}], [${dice [3]}] , " echo " [${dice[4]}] and [${dice [5]}]" echo -n "Your new roll: [${dice[1]}] , [${dice [2]}] , [${dice [3]}] , echo " [${dice [4]} ] and [${dice [5]} ]" exit 0 die in ordinal value and let the player enter the appropri¬ ate number to indicate that it should be rerolled. Urn, let me show you what I mean: echo -n "Reroll which dice? read answer for reroll in $answer do echo "Requested: $reroll" done Here, you might specify that you want die 1 and 3 rerolled by typing in 1 3. Tweaking this just a bit, the for loop then can test for the validity of each entry: for reroll in Sanswer Let's run it once to see what's happening, and then next month, we'll start working on the actual game itself, rather than just the dice rolls: $ ./yahtzee.sh You rolled [2] , [6] , [5] , [2] and [1] Reroll which dice? 235 Your new roll: [2], [2], [4], [2] and [5] Yes, I snuck in the notation of having the dice values shown within square brackets just for visual appearance. It makes the echo statements a bit more confusing, as you can see just a bit earlier, but the output is more attractive. ■ Dave Taylor is a 26-year veteran of UNIX, creator of The Elm Mail System, and most recently author of both the best-selling Wicked Cool Shell Scripts and Teach Yourself Unix in 24 Hours, among his 16 technical books. His main Web site is at www.intuitive.com. and he also offers up tech support atAskDaveTaylor.com. The yahtzee.sh script is also available on the LJ FTP site at ftp.linuxjournal.com/pub/lj/listings/issuel 62/9819.tgz 36 | October 2007 www.linuxjournal.com Root Access: Providing the control you need. Advanced FairShare Technology: Better resource management means better performance. Support That's Actually Supportive: Award-winning support provided by system administrators. 1 jfk f' Ha*' HOSTING IS LINUX WITH A LINEAGE. Verio Linux® VPS and MPS: Best of Breed. At Verio, we have a long-running commitment to open source, dating back to our early work with FreeBSD. Now, as the pioneer in virtual private server (VPS) technology and as a hosting provider backed by the financial resources of the world's largest telecommunications company, we bring something extra to Linux: reliability. To learn more about Verio Linux VPS or Verio Linux MPS, or to register for a free test-drive, call 1 -877-837-4654 or visit www.verio.com/linuxlineage. Verio and the Verio logo are trademarks and/or service marks of Verio Inc. in the United States and other countries. Linux® is the registered trademark of Linus Torvalds in the U.S. and other countries. The mark FreeBSD is a registered trademark of The FreeBSD Foundation and is used by Verio Inc. with the permission of The FreeBSD Foundation. All other names are trademarks or registered marks of their respective owners. ©2007 Verio Inc. All rights reserved. Build on us. VERIO An NTTCommunications Company COLUMNS BEACHHEAD Get FOSSED David Trask leverages the flexibility and power of FOSS to create a waste-not-want-not approach to educational computing. JON "MADDOG" HALL This month, I step outside my normal persona to describe a small conference named FOSSED (www.fossed.net), I attended two sessions of FOSSED, one in June 2007 and one in July 2007, which finished just two days before I wrote this article. David Trask, a friend of mine, initiated FOSSED five years ago. David is an elementary schoolteacher in Maine, and he is "student focused". He often reminds people that his customers are the students, and in David's case, these are second- and third-grade students, eight to ten years old. David, like many teachers, always is asked to "do more with less", and in the course of trying to do this, he discovered free and open-source software. David started using FOSS in his own school. He became a fan of the Linux Terminal Server Project (LTSP, www.ltsp.org) and started re-deploying cast-off equip¬ ment to create more computing nodes for his students. David also is a student of human nature, and as he started to deploy more and more of these systems for his own students, and as they started doing more and more in his own classes, he offered some of these computers to other teachers. David also is adept in telling stories about his young charges and how easily they adapt to FOSS while older people complain how hard it is. After a while, David began sending out e-mail messages regarding his successes to a mailing list read by Maine educators involved with technology. Eventually, David realized enough people were interested in this topic that he decided to have a few seminars and actually demonstrate the capabilities. After a few of these seminars, David started talking about the possibility of having a conference specifically about the use of FOSS in grade-school education. So, five years ago, David and another FOSS enthusiast held the first educational FOSS conference at Gould Academy (a residential high school in Maine) and called the conference NELS, for North East Linux Symposium. They later realized that Linux was a small (although important) part of FOSS educational software, so they renamed the conference FOSSED. In June 2007, right after school let out, 70 elementary-, middle- and high-school teachers and technologists descended on Gould Academy. Out of the backs of cars and trucks came systems removed from school labs and homes, networking gear and personal laptops for taking notes. In addition, several sponsors supplied a dozen or more thin-client systems and a server, so people could use them during the event. [Note to sponsors of other events like this: the thin clients went on sale after the event at a "special price" to attendees.] By this time, David had picked up several other helpers. Matt Oquist, the founder of Software Freedom Day (www.softwarefreedomday.org), who also is a consultant to various school systems in the use of free software, helps David in the planning and execution of logistics and teaches some classes. Bill Sconce, a consultant and scripting wizard, gives courses in scripting languages for more-advanced participants to show how to maintain many systems using a shell script, Python and other scripting languages. Bryant Patton, a longtime advocate of computers in education and the founder of the National Center for Open Source and Education in Vermont, also helps out. But, David Trask remains the driving force and cheerleader, and for the past two years, the University of New Hampshire (UNH) has held a second session of this conference with almost as many attendees as the sessions in Maine. Five years after David started the conference, the amount of software that exists for educational use is very impressive. A couple seminars are centered around the use of Moodle to plan and present classroom material. One teacher volunteered to talk about Web 2.0 tools and show the other teachers how some of the Web's new features could be used to stimulate student creativity. At the same time, she discussed doing this in a safe way and getting around the advertising bombardment that sometimes accompanies gratis sites. Although a core component of the conference is LTSP usage and administration, it also has branched out to discussions on LDAP for ease of administration, IP Cop (for setting up firewalls), creating software packages (think .deb and .rpm) and a variety of other system-administration topics. It was interesting to see how one moment the person sitting next to me was a student in a class on how to use VMware effectively, and the next moment that same person was teaching a course on some other aspect of free software—teachers teaching teachers. Sometimes the thing most obvious to you may be the biggest revelation to another person. I had not used VMware since its early days when it was difficult to install and get working. At the conference, I "installed" VMware (actually it was a package already done for Ubuntu), entered the key, installed an Edubuntu server on it and booted a thin client running off the virtualized server on my notebook—all of this while I was still editing and sending e-mail through my notebook's wireless card. I am now quite a fan of VMware, and I am sorry I haven't spent more time with it over the years. On the other hand, when the teachers mentioned that 38 | October 2007 www.linuxjournal.com there was so much software on SourceForge that they didn't have time to evaluate it all, I suggested that they ask students to find, test and select software the class might want to use and present their results and reasoning to the class—the 21st-century version of the 19th-century book report. A stunned silence fell over the room. At the UNH sessions, a group of high-school students and their advisers came up from the Arlington, Virginia, school system (an 11-hour ride by car in two vans) to demonstrate a project called CanDo that they had been creating. Then, after getting feedback from the teachers about new features and GUI changes, the students went into a two-day (and sometimes night) hacking session to implement those changes. It warmed my heart to see that the group of students was a diverse mixture of young men and women, different races and nationalities, and they all seemed to get along fine. This was the second year that they participated in the conference, and it was good to see some of the same faces return. Finally, the teachers listened to and gave feedback to several of the vendors who had open-source products used at the event, and to one vendor who was struggling with whether the company should go open source and the consequences of doing that. Direct feedback from customers is usually a good thing to have. Many things impressed me about this conference, but one of them was the goodwill and camaraderie that man¬ aged to come through all the time—people working with each other and having fun learning. The organizers try very hard to have a comfortable venue and a relaxed schedule, and although it's too late to participate in a FOSSED event this year, you can start thinking about participating in next year's event, or (better yet) start thinking about creating a similar event in your own region of the world. You can see a lot of what was done last year at www.fossed.net and participate in the FOSSED blog at fossed.blogspot.com. ■ Jon “maddog” Hall is the Executive Director of Linux International (www.li.org), a nonprofit association of end users who wish to support and promote the Linux operating system. During his career in commercial computing, which started in 1969, Mr Hall has been a programmer, systems designer, systems administrator, product manager, technical marketing manager and educator. He has worked for such companies as Western Electric Corporation. Aetna Life and Casualty. Bell Laboratories, Digital Equipment Corporation, VA Linux Systems and SGI. He is now an independent consultant in Free and Open Source Software (FOSS) Business and Technical issues. The Leading Management, Strategy & Policy Event for Today's Security Professionals Risk & Metrics Gov, Law & Compliance Attacks & Countermeasures Awareness Training Incident Response Apps & Endpoints Rising Threats CSI 2007 will show you how to navigate the management and strategic issues that form the core of a well-protected organization. NOVEMBER 3-9, 2007 | ARLINGTON, VA COLUMNS LINUX FOR SUITS DOC SEARLS Atlas: Hoisting a New World of Search There is history here. We first covered Jabber in the September 2000 issue of Linux Journal, more than seven years ago. At the time, Jeremie Miller, who invented Jabber, told me at least five years would pass before Jabber's protocol—later dubbed XMPP and approved by the IETF in 2003—would establish itself as a de facto standard. He was right. We've stayed in touch over the years, as Jeremie's interests have spread outward from messaging and presence to other subjects—especially search. I guess it was about two years ago that he began to question whether search needed to be a single-source thing (for example, Google or Yahoo). That was when he also started talking about his new search project, called Atlas. Whenever I'd ask him what he was working on, he'd reply, "Atlas". I was all for it, because I've felt from the start that search engines are essentially kluges meant to over¬ come a directory deficiency in the Web itself. I've also thought that, although there was good to be found in the chaotic nature of everything to the right of the first single slash in every URL on the Web, there was something inherently wrong about relying on massive commercial advertising-powered search engines—with their bots and crawlers and proprietary weighting algorithms chugging through constantly updated indexes stored in hundreds of thousands of servers— just to find stuff. And, although I agree with David Weinberger that Everything Is Miscellaneous (the title of his excellent new book), I don't like relying on Google, Yahoo or MSN alone (usually just Google) to tell me what I mean when I search for something. Nothing matters more than meaning, and I don't like seeing it supplied by what Jeremie calls a "text box dictatorship". In a May 28, 2007 blog post, he asks: Why in such an advanced civilization have we become Knowledge Peasants who are so easily placated by the black magic of our Goovernor? Am I the only one wondering why these commercial boxes own such an important social function: what everything means? The answer, he says, is: Open open open! Open source, open distributed grids, open algorithms, open rankings, open networks of people cooperating to provide resources. The future of search is in open cooperation (and competition) based on a Meaning Economy—create meaning, exchange meaning, serve meaning. My vision begins with an open protocol, allowing independent networks of search functions (crawl¬ ing, indexing, ranking, serving, etc.) to peer and interop. All relationships between these networks are always fully transparent and openly published. Networks exchange knowledge between them, each adding new meaning to the information, each of them responsible for the reputations of their participants and peers. This is the very foundation of a Meaning Economy. Tomorrow now has a meaning that we can all help build. Jeremie hasn't been the only one on the open search case. Jimmy Wales, prime mover behind Wikipedia, attracted attention in December of last year when he said, "I want to create a completely transparent, open-source, freely licensed search engine"—as part of Wikia.org, a community develop¬ ment companion to Wikipedia. In a December 29, 2006 interview (www.wired.com/techbiz/it/news/ 2006/12/72376?currentPage=2), Wired asked him for specifics about that. Jimmy's reply was, "We don't know. That's something that's really very open-ended at this moment. It's really up to the community, and I suspect that there won't be a one-size-fits-all answer. It will depend on the topic and the type of search being conducted." Subsequent interviews were similarly speculative and open-ended. Then, on May 1, 2007, came news that Jeremie was joining the Wikia Project. In a prepared statement (news.com.com/2100-1032_3-6180379.html), Jimmy Wales said, "Jeremie is a brilliant thinker and a natural fit to help revolutionize the world of search....I believe Internet search is currently broken, and the way to fix it is to build a community whose mission is to develop a search platform that is open and totally transparent." Atlas was unveiled in a post to a list on July 5, 2007 (lists.wikia.eom/pipermail/atlas-l/2007-July/000000.html). In that post, Jeremie said his "large vision" is "enabling search to become a part of the Internet's infrastructure. Building on Atlas as an open protocol, search can become a fully distributed and interoperable world-wide community. All of the participants can interact openly and in any role where they believe they can add value to the network." As for architecture, he offered this: 40 October 2007 www.linuxjournal.com There are three primary roles within Atlas: ■ Factory—responsible to the content. ■ Collector—responsible to the keyword. ■ Broker—responsible to the Searcher. Each of these actors must interact with the others to complete any search request. Any two roles could be performed by a single entity (whereas if all three are performed by one entity, the result would be a traditional, monolithic search engine). A Factory is akin to a crawler in today's search engines. An Atlas Factory must fetch and process the content as intelligently as possible, performing analysis (such as Natural Language Processing) and normaliz¬ ing it into distinct units. A Factory shares its highly refined and pro¬ cessed output with one or more Collectors based on who they believe is best utilizing it. A Collector absorbs and indexes output from one or more Factories, with one primary goal: ranking. An Atlas Collector must provide the most intelligent ranking and relationship analysis possible. A Collector has to compete for the output of a Factory, as well as compete to provide the best ranking quality for Brokers. A Broker must provide a Searcher with the best possible results. It does so by combining diverse ranking results from Collectors and also by retrieving content from the original Factories. This last step, a Broker interacting with a Factory, is critical to maintaining a balanced ecosys¬ tem. All Factories must be aware of and approve how their results are being used and by whom. Reputation and reward is bi-directional between all parties (Factory-Collector, Collector-Broker and Broker-Factory). Each entity may choose to interact on principle (free, Commons), attribution (results provided by), or commercially (as a paid service). The Atlas protocol is purely a facilitator and does not restrict how the relationships between any entities are formed. In considering these motives for the various entities, it's likely that the free-based networks will tend to become more specialized, commercial ones will compete on quality, and attribution-based networks will mature in both directions. This simple yet powerful division of roles, responsibilities and relation¬ ships will result in a distributed economic foundation for an Internet Search Infrastructure. The wire protocol and further definition of the interactions between these entities is openly evolving; anyone interested is welcomed to join the discussions and see the initial proposals at lists.wikia.com/mailman/listinfo/ atlas-1 over the coming weeks. As a kind of gauntlet, Jeremie threw down a summary challenge, "Nobody will beat Google, but EVERYBODY will." Vigorous discussion ensued, as a rapidly growing community of developers began getting into what Jeremie calls "the dirty work of building it openly now". As that work began, I asked him if it would be cool to flow some of our conversation over here to Linux Journal. He said sure, so here it is. DS: There is always this tug between monolith and polylith. The irony of the Net as a Giant Zero (world of ends) is that it is entirely polylithic—or wants to be. Centralizing a future polylithic protocol into a monolithic service is one way it starts. But the end state is polylithic. JM: I agree, it's inevitable. It's the being of the Net itself that ultimately demands it, but Google is fighting to be a monolith for as long as possible...and that's fine, they'll embrace Atlas when they see it providing value. DS: In your announcement [above] I see the seeds of a credit-where-due-based economic system—one in which we might obtain finders fees, or something like that, as value is given for service performed. JM: The attribution-based model, yes. Absolutely the middle man needs to be Linux Laptops Starting at $799 Linux Desktops Starting at $375 Linux Servers Starting at $899 DON'T BE SQUARE! CET CUBED! Technologies 309.34.CUBED shoprcubedxom COLUMNS LINUX FOR SUITS involved in the transaction. Atlas doesn't flow the money, but it does flow the information and provide a framework. The same with advertising. Really, contextual ads are very helpful. I rely on them as a tool when using Google search. And in fact, that model is the best form of fighting Web spam. How the system works, and who is involved in the flow of information, is completely transparent, so the three actors—a Factory, a Collector and a Broker—are all involved in providing a search result. A Broker works on behalf of the Searcher. They have relationships with the appropriate Collectors, plural, and perform the queries— assembling all the relevant "Knuggets" they get back from the Collectors, valuing them based on whatever metrics they want, including talking to a "sponsored" Collector who serves only commercial results, any "local" Collectors for regional areas, and so on. Competition is fierce when anyone can be a Broker for almost no cost other than relationships. So, a Collector has one job, provides relevant results, and has to compete with anyone else to do so. And, it can judge the relevancy via any algorithmic, human-reputation-based, or combination. DS: Open source at the production end has always been a meritocracy. Seems to me the equivalent with Atlas to "show me the code" is "show me the results". Or at least, "show me the relevancy". No? JM: The Factory is managing commodity access to the refined content, doing all the work of normalizing the Web. Search results are just merit: who has the best. And a Broker going to many sources, many Collectors (there can be lots of them) gains lots of merits in different forms. So foxmarks has a great database of deep links that are very important to people. That way Mitch can serve high-quality results but only for certain categories of queries. And Wikipedia can serve another category of queries with high relevancy—as can local yellow-page-style systems, as can social networks for people queries. DS: I see implicit in this a respect for the snowballing nature of knowledge, both for individuals and for groups. To be human is to grow what one knows. Authority is the right we grant certain others to contribute to what we know—and to change us in the process. Knowing more makes me different. As has been said elsewhere, "we are all authors of each other". JM: Yes, the fundamental unit of Atlas is a "Knugget", a Knowledge Nugget essentially, a search result. A Factory adds value based on what it knows about the content; a Collector adds value based on what it knows about keywords and ranking Knuggets; and a Broker adds value based on what Collectors it knows and what value they provide in aggregate. DS: Wikipedia, in growing its own relevance, is an interesting example. I've been looking at radio stations and Webcasters. Wikipedia on the whole is a great source of info, but it's far from complete. And, it needs a better way to stay complete than just relying on narrow subject obsessives to stay on top of the current narrative. Search results that feed into better Knuggets that turn into better Wikipedia entries should be a Good Thing, no? JM: Yes. DS: Is a Knugget "something somebody wants to know"? I like the word. How about if it's a combination of keywords that may change over time? JM: A Knugget is one unit of context, as I define it. It may be a title and a link; it may be a sentence saying some¬ thing about a noun; it might be a row from a table of things. It's human-defined and, therefore, very fuzzy by its very nature. It's "What would a human recognize and make some sense of, out of context of anything other than what's contained inside of it?" The Web is human, not machine, and Atlas reflects that. DS: I like contextuality. The summit of Mt. Everest can be an elevation, a sum of climbers, a single fact (such as, it is marine limestone). JM: Yes. The very nature of Atlas is to demand that a Factory produces the best Knuggets, that a Factory "understands" the content as best as it can. It is a model that rewards human understanding and value first. All derivative knowledge is built atop that foundation, and we can reduce all of these inflated DB/schema disasters, which all serve the machine first. Atlas works in the same way that the Internet served people first, and servers/data centers second—and the Web content serves people first and software second. Search-as-lnternet-infrastructure must serve people first. DS: Yes. I think there is a Static (traditional Google) vs. Live (human, now, evolving) distinction here. Allow me to quote myself at a bit of length: "And as for live feeds of Knuggets as they are produced, any content provider can get the most value by generat¬ ing these feeds of Knuggets into Collectors they trust, search results can be instantly rewarded, a Searcher can find that breaking-news article immediately." One becomes an Atlas Broker just by being involved, I would gather. JM: Just by searching, and knowing whom to ask for results. DS: How do you want to seed this thing? Where does actual use start first? JM: An Atlas Broker should be living here in my IM client, using all this great context, LOCALLY, to search my IM his¬ tory as well as provide the best relevant search results from any content store I want it to. DS: Who is the Broker? 42 | October 2007 www.linuxjournal.com The Straight MkPeople SM SINCE 1991 ABERDEEN SUPERSIZE YOUR STORAGE 1TB DRIVES & QUAD-CORES 1U 14 Inch Depth Mini Rack • Up to two Dual-Core or Quad-Core Intel® Xeon® processors with up to 1333FSB and 8MB Cache • Intel® 5000V “Blackford VS” Chipset with 64-Bit Support • Up to 16GB 667/533MHz Fully Buffered ECC DDR2 SDRAM • Up to 1TB SATA Hard Drive • 520W AC Power Supply w/PFC • 5-Year Warranty Starting at * 1,339 1U Short Depth 4 SATA 3TB • Single Dual-Core or Quad-Core Intel Xeon processor with up to 1066FSB and 8MB Cache • Intel 3000 “Mukilteo-2” chipset with 64-Bit Support • Up to 8GB 667/533MHz DDR2 Memory • Up to 4 x 750GB (3TB) Hot-Swap SATA Hard Drives • 300W AC Power Supply w/PFC • 5-Year Warranty Starting at * 1,079 1U Twin Node “Blade Killer” • Up to two Dual-Core or Quad-Core Intel Xeon processors per node • Twin Nodes allows for up to 4 processors and 16 cores in 1U • Up to 32GB 667/533MHz Fully Buffered ECC DDR2 per node • Up to 2 x 1TB (2TB) Hot-Swap SATA Hard Drives per node • Optional Infiniband Controller per node • 5-Year Warranty Starting at * 2,669 1U Quad-Core Universal I/O 4TB • Up to two Dual-Core or Quad-Core Intel Xeon processors • Intel 5000P “Blackford” Chipset with 64-Bit Support • Up to 32GB 667/533MHz Fully Buffered ECC DDR2 SDRAM • Up to 4 x 1TB (4TB) Hot-Swap SATA Hard Drives • Universal 1/0 allows for 3 expansion cards in 1U • 650W 1 +1 Hot-swap Redundant Power w/PFC • 5-Year Warranty Starting at * 1,949 2U Quad-Core 8 SATA/SCSI/SAS 8TB • Up to two Dual-Core or Quad-Core Intel Xeon processors • Intel 5000P “Blackford” Chipset with 64-Bit Support • Up to 64GB 667/533MHz Fully Buffered ECC DDR2 SDRAM • Up to 8 x 1TB (8TB) Hot-Swap SATA Hard Drives • Up to 8 SCSI/SAS Hard Drives available (Stirling 228S/228Q) • 700W High-efficiency Redundant Power Supply • 5-Year Warranty Starting at ^2^339 3U Quad-Core 16 SATA 16TB • Up to two Dual-Core or Quad-Core Intel Xeon processors with up to 1333FSB and 8MB Cache • Intel 5000P “Blackford” Chipset with 64-Bit Support • Up to 32GB 667/533MHz Fully Buffered ECC DDR2 SDRAM • Up to 16 x 1TB (16TB) Hot-Swap SATA Hard Drives • 650W 2+1 Redundant Hot-Swap Power Supply • 5-Year Warranty Starting at ^ 3^8 9 9 BERDEEN STIRLING X526 BERDEEN STIRLING X633 5U Quad-Core 24 SATA 24TB • Up to two Dual-Core or Quad-Core Intel Xeon processors • Intel 5000P “Blackford” Chipset with 64-Bit Support • Up to 32GB 667/533MHz Fully Buffered ECC DDR2 SDRAM • Up to 24 x 1TB (24TB) Hot-Swap SATA Hard Drives • 2 x Internal SATA Hard Drives for OS • 950W 3+1 Redundant Hot-Swap Power Supply • 5-Year Warranty Starting at * 5,149 6U Quad-Core 32 SATA 32TB • Up to two Dual-Core or Quad-Core Intel Xeon processors • Intel 5000P “Blackford” Chipset with 64-Bit Support • Up to 32GB 667/533MHz Fully Buffered ECC DDR2 SDRAM • Up to 32 x 1TB (32TB) Hot-Swap SATA Hard Drives • 2 x Rear Hot Swap SATA Hard Drives for OS • 1350W 3+1 Redundant Hot-Swap Power Supply • 5-Year Warranty Starting at * 6,699 Intel, Intel Logo, Intel Inside, Intel Inside Logo, Pentium, Xeon, and Xeon Inside are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. For terms and conditions, please see www.aberdeeninc.com/abpoly/abterms.htm. Ij020 Xeon* inside n Quad-core. Unmatched. 888-297-7409 www.aberdeeninc.com/Ij020 COLUMNS LINUX FOR SUITS JM: The Broker is a human. The Broker is just a service that uses as much context as you, the Searcher, wants to give it, and it talks to many Collectors to merge/provide the best results. DS: So, how are you seeding this thing? JM: I'm already starting to set up Atlas Factories. We may announce a contest to build open-source Collectors that can do different/cool things with, say, the Internet Archive. Once Atlas starts to breathe on its own, and can publish that archive either openly or gated (attribution-based or paid), or if it can rank Web results better for a certain class of queries, it can use that data as a Collector and offer that again, openly or gated. But that's speculative and premature. It's all wide open at this stage. DS: Outstanding. I know some investors have leaned on Technorati to blow away the archives because that's not what they search. Yet. I guess I need to get a sense of what a search might look like, and how it would come up with stuff that's different from Google's static search and Technorati's live (chrono) search. Would I go to atlas.wikia.org to search? Or to...where? Or is that question too static or site-based? JM: I don't have a "static" presence for Atlas yet. Kind of refusing even to do that for whatever reasons. Just a link to the mailing list for now is all there is. Someday there will be a presence, but the discussion is more important right now. I like it that all people can do at the moment is discuss. DS: I'm just looking at how to help people conceive What It Is. Is it a site? A service that other sites, or even IM systems, or cell-phone apps, can use? JM: Atlas is an idea, a model and, ultimately, a communica¬ tion system between two people: the one that wants to learn and the one that wants to teach/share. It's just another communication platform, but the people talking don't know each other yet. Like all good Internet systems, it will live under the hood, behind text input boxes everywhere. DS: In that respect, it's more like the Jabber "plat¬ form" than the AIM or Skype "platforms". JM: Yep, in that Google, Yahoo and Ask are the IM silos, and Atlas is the distributed/open Jabber model. DS: Good. I get that. Is Atlas code that sits some¬ where and is given a bunch of stuff to look at? If so, where does it live? Is there a drawing we can use? Some kind of simple whiteboarding? JM: Where stuff is almost doesn't matter. As far as visualiza¬ tions go, the "logical" one is really Factory^Collector^Broker, but the technical one is more of a triangle, as the Broker talks to the Factory in the end to get the Knuggets...but basically no, there's nothing visual yet. A Factory is going to "look" like a pile of search results ordered based on the content source alone. A Collector will aggregate/order them. And a Broker will aggregate from lots of Collectors the ordered results, merging them, getting the snippets from the Factories they came from, and presenting them. By the way, someone made the first visualization of the model (Figure 1). i c t .iwi th* TOh an* , am ,*I ,n t ! ft* cgritflntf caAs^otiied conltnls Factory Collector Figure 1. Visualization of the Atlas Model (from search.wikia.com/ wiki/Atlas) DS: Where would they live? The actors...Factory, Collector, Broker...or the code for it all? JM: Oh, anyone can run any of them. There will be open- source projects to provide each of them. But, there should be thousands of each, running everywhere around the world. Just like Jabber, e-mail and Web servers. They just talk to each other with a protocol. There can be locale- based specific instances, ones for different languages, ones for types of content (images, videos), ones for topics (gaming, finance)—whatever people want to do/specialize in. People can run them for whatever reasons they want. A Broker is really the endpoint, so the nature of the search has the Broker engaging the relevant Collectors. A Broker is doing what the word really means, brokering your search to lots of sources for the best results. A Broker is what would likely be built in to your browser or whatever is driving an input box anywhere. DS: I like the way it maps to the real world. JM: It doesn't force an operational model, and it just goes with whatever motives people have to run it. The good part is that it will work only if people find it valuable enough to run it; those are the best kinds of systems. So is all of this clear as mud? DS: Good mud! JM: Thanks. For more, look up Jeremie and Atlas on Google. Until you can do the reversed Doc Searls is Senior Editor of Linux Journal. He is also a Visiting Scholar at the University of California at Santa Barbara and a Fellow with the Berkman Center for Internet and Society at Harvard University. 44 | October 2007 www.linuxjournal.com Much Better TCO than 1U Servers • Features Intel® Xeon® and AMD Opteron® DP/MP • Holds up to 10/14 server blades • Six enclosures fit a 42U rack • 90%+ High-Efficiency, 3+1 redundant power supplies • Chassis management modules • Gigabit Ethernet switch modules • Gigabit Ethernet pass-thru modules • InfiniBand switch modules (20 Gb/s x2) Application-Optimized for: Enterprises, Financial Services, Databases, Data Centers Science Labs, HPC, Personal Super Computer www.supermicro.com ■“ _ * »■ CMM IPMI View CMM Web-based GUI Him s in r. 4x DDR Infiniband switch Intel Processor Blade w/ 3.5 in. Hot-swappable Drive Bays AMAX Arrow Electronics ASI Bell Micro Ingram Micro MA LABS 1-800-800-6328 1-888-427-2250 1-800-2000-ASI 1-800-232-9920 1-800-456-8000 1-408-941-0808 www.amax.com www.arrownacp.com www.asipartner.com www.bellmicro.com www.ingrammicro.com www.malabs.com Synnex 1-800-756-5974 www.synnex.com Tech Data 1-800-237-8931 www.techdata.com © 2007 Super Micro Computer, Inc. Specifications subject to change without notice. All other brands and names are the property of their respective owners. Intel, the Intel logo, Intel inside, the Intel Inside logo, Intel Xeon are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. NEW PRODUCTS r AML's MT7570 Vehicle Mount Terminal If your forklift needs a Linux-driven data-capture device, AML hopes you'll use its new MT7570 vehicle mount terminal. The MT7570 is designed for "real-time receiving, put-away, picking and shipping applications in harsh industrial environ¬ ments", and it integrates securely into existing wireless networks. Full USB and RS-232 serial connections, as well as optional Bluetooth WPAN communications, provide connectivity to peripheral devices, such as bar-code scanners and print¬ ers. Construction is rugged. The MT7570 has a bright display for dimly lit envi¬ ronments and can withstand dust and water deluges. The device is available with either embedded Linux or Windows XPe, and both systems include terminal emulators (VT100/220, TN5250, TN3270), Web browsers and a Skype client. www.amltd.com ansur Michele Davis and Jon Phillips' Learning PHP & MySQL, 2nd Edition (O'Reilly Media) In our enlightened community, PHP and MySQL are becoming the typical tag team for developing database-driven Web development. If this is your calling, pick up the new 2nd edition of Davis and Phillips' book Learning PHP and MySQL, published by O'Reilly. Intended for newcomers to the technologies, the book teaches both the PHP language and the MySQL database separately and then shows how to merge the two to generate dynamic content. It also contains content on XHTML, error handling, security, HTTP authentication and more. www.oreilly.com Silicon Mechanics' Bladeform 8100 Series Blade Server Platform Silicon Mechanics recently rolled out its new Bladeform 8100 Series of blade servers. The firm describes the line as "a family of modular computing products designed to address a wide range of high-density computing challenges by allowing multiple servers to be con¬ tained within one easy-to-manage system." Series components include the blade server enclosure, the 8110 server blade (dual Intel Xeon), modular networking and interconnect components. Some of the key features include a modular enclosure with support for up to ten server blades, up to four redundant load-balancing power modules, 90%+ efficient power supplies, up to 2GB Ethernet switches with ten external ports each, InfiniBand expansion adapters and switch support, and remote management capabilities. www.siliconmechanics.com HPC Systems' HiPerStor NAS Product Line We've been informed of HPC Systems' HiPerStor, a new line of network- attached storage products. The line is targeted at three different product seg¬ ments, namely SOHO/home, SMB and SMB with advanced needs. The line also features iSCSI technology, upgrade to InfiniBand, TOE NICs or 10GbE NIC, integrated volume replication and snapshot, support for disk encryption, secure Web-based management, a range of user-authentication options and more. www.hpcsystems.com 46 | October 2007 www.linuxjournal.com 1 NEW PRODUCTS Integrated Computer Solutions' UIM/X We bid a warm Linux-community welcome to Integrated Computer Solutions, which recently released version 3.1.1 of UIM/X, a client/server application-development tool that now also runs on Linux. UIM/X enables developers to build Motif GUIs "in a fraction of the time it takes by hand", say the folks at Integrated. They also claim that UIM enhances programmer productivity by enabling the creation, modification, testing and code generation for the user interface portion of an application with a single tool. UIM/X supports the most current version of Motif (2.x) and runs on Solaris, HP-UX and Red Hat Linux. www.ics.com — HjspSSSiM fdiiJU y £ «j EaSIfp , MHiUMB1 Mil Storix, Inc.'s SBAdmin In other news from LinuxWorld, Storix, Inc., released SBAdmin v6.2, touted as the first backup and system recovery solution to integrate with IBM's Tivoli Storage Manager (TSM). SBAdmin complements TSM by expanding on its features and capabilities for Linux and AIX systems. SBAdmin also writes directly to the TSM server, which cuts extraneous steps, saves time and increases reliability. In addition, the application provides TSM users with disaster-recovery capabilities, including a feature called Adaptable System Recovery, which enables system restore to the same or dissimilar hardware. www.storix.com 64 Studio Although I fear Dave Phillips will scoop me on his "Studio Dave" blog, let it be known that 64 Studio 2.0 Electric has been released. 64 Studio is "a GNU/Linux distribution tailor-made for digital content creation, including audio, video, graphics and publishing tools." A remix of Debian, 64 Studio comes in both 64- and 32-bit flavors, allowing it to run on nearly all PC hardware. The distro combines the stability and quality of Debian Etch with a specialized real-time preemption kernel and the latest creative tools demanded by multimedia artists. The developers aim to rocket you from blank hard drive to a fully hardware-optimized and usable creative desktop in just half an hour. www.64studio.com blackduck Black Duck's ProtexlP At LinuxWorld San Francisco, Black Duck Software announced version 4.4 of the firm's protexlP/development, "a platform that helps companies govern how their software assets are created, managed and licensed." ProtexlP helps developers and legal counsel in managing the use of code from open-source projects that have both decided to switch to GPLv3 explicitly and those that have decided not to switch. It also includes an enhanced KnowledgeBase, a library of open-source and vendor-added code software components that includes detailed licensing information for more than 140,000 components. www.blackducksoftware.com Please send information about releases of Linux-related products to James Gray at newproducts@linuxjournal.com or New Products c/o Linux Journal 1752 NW Market Street, #200, Seattle, WA 98107. Submissions are edited for length and content. www.linuxjournal.com October 2007 | 47 The Tornado M20 Phone and Digital Media Center Why settle for VoIP when you can pack digital media in the same package? MIKE DIEHL W hen I considered evaluating SysMaster's Tornado M20 Media Center, I thought I was simply evaluat¬ ing a video/voice IP telephone. Imagine my surprise when I discovered that it also could handle IPTV, streaming audio, stored graphics, IM, e-mail, RSS news and weather! I thought all it needed was a built-in calculator. Then, I read the Owner's Manual and discovered that it does have a calculator function. Wow! All this inside a box the size of an average speaker phone, and it even has space for 10Mb LAN, 802.1a Wi-Fi and a 3.5 color video screen. For about $250 US, this device seems to be able to do everything my MythTv system can do, but it's much smaller! Enough hype. Let's talk about practical issues. As soon as I unboxed the unit, I was immediately struck by how heavy it was. This is not a phone you'll be dragging across your desk every time you lift the handset. When you put this unit on your desk, it's going to stay where you put it, and it's going to look nice sitting there. The M20 is a very clean, professional-looking phone. Appearances are more important than you might think. I once was involved in a Voice-over-IP (VoIP) evaluation project where we set up a VoIP system and deployed a few phones for potential users. The prevailing comments we received from the users were that they liked the system. They liked the sound quality. They hated the phone. They said it looked too much like a toy. Obviously, we ended up taking another direction. The M20 is black and silver-tone with clearly labeled buttons. There are no silly icons next to each button to make you guess what the button does. The handset, though per¬ haps a bit light, is comfortable to use. The color display and pivoting camera on the side of the unit tease functionality beyond mere voice communication. The M20 is a four-line VoIP telephone that uses the SIP protocol to make and receive voice and video phone calls. Video conferencing uses the H.264 protocol. For voice calls, the device can use the G.711, G.729 and G.726-32 codecs. For video calls, the unit supports H.263. Streaming media uses MP3 and MPEG-4. The unit's network configuration is done via DHCP, PPPoE or can be done manually. The rest of the configuration is done with a Web interface or a central provisioning mechanism via HTTP. As far as I can tell, there is no closed or proprietary functionality in this unit, which is important to me as a Linux and open-source advocate. Because there are many voice-only VoIP phones out there, most of which are considerably less expensive, I opted not even to bother testing the voice-only capability and jumped straight to the main event, video. So, I asked one of the technical-support staff members at SysMaster to initiate a video call to me. When the M20 receives a call, it indicates that it is a video call by displaying a V next to the caller-ID information on the display. If you want to establish a video call, you simply press the VIDEO button on the phone. On the other hand, if you want to establish a voice-only call, you pick up the handset, just like a regular phone, and the video func¬ tion is disabled. Placing a voice or video call is done similarly. Before we started our video call, the technician made sure I understood that the quality of the call was limited by the available bandwidth. I was using a residential DSL connection, so that would be the limiting factor. However, when we made the call, I didn't feel limited at all. I was able to see the technician clearly, and in color—the color was a bit washed out though. Later, I discovered that the video camera was quite tunable, and that the camera's brightness, sensitivity and hue were all adjustable. I suspect his camera was just poorly adjusted. When I was playing with the phone earlier that morning, I was able to use the camera and see myself on the display, in full, vivid color. As scary as that was, it wasn't the phone's fault. 48 | October 2007 www.linuxjournal.com As with any CCD camera with a slow frame rate, objects that are moving quickly tend to blur, but normal speaking and moving looked natural. However, when the technician waved his hand across the field of view to demonstrate the effect, the blurring was apparent. The default frame rate of 3-5 frames per second can be adjusted, but it seemed adequate for normal usage. Adjusting it much higher probably would have exhausted my available bandwidth and caused the call quality to drop sharply. Even though we had a full video stream between us, the sound quality was still quite usable. The sound quality wasn't quite up to the level of the Polycom 501 I use at home, but in my opinion, nothing is. Even so, it was better than many VoIP phones I've used in the past. The M20 supports all of the normal call functions, such as call forwarding, call transfer, hold, do-not-disturb, message waiting and voice mail. The phone supports Network Address Translation (NAT) as well as multiple server registrations. In addition to normal call features, the M20 can function as an alarm clock by sounding the phone ringer at preselected dates and times. But, you can take it further than that. The device also can be configured to call an external phone number at a preselected time. The device can store up to 80 voice-mail messages locally. The voice-mail system amounts to an internal answering machine, but at least it doesn't require a centralized server. Recorded messages are easy to retrieve via the menu or with a Web browser. The company's literature boasts advanced IVR (Interactive Voice Response) capability, and I think you'll agree I've got enough to write about as it is without getting into that function in any depth. I'm told that the devices will auto-discover each other on the same network and establish a peer-to-peer telephone PBX configuration. So, it's a pretty well-rounded SIP phone. After hanging up with the support technician, I decided to watch some TV—IPTV, that is. Accessing the IPTV feature is easy using the on-screen menu. Once activated, the IPTV feature dis¬ plays a list of available channels. A channel is selected by pressing the up and down buttons on the phone and then by pressing the OK button to begin viewing. The color display is simply stunning! I watched a soccer game on the unit; the field looked green, and the players didn't. Flesh tones were realistic. Motion was smooth without any hint of ghosting. Once in a while, I noticed some video artifacts, but that probably was due to bandwidth limita¬ tions or packet loss. Using the built-in PVR function is much like programming a VCR. You select the date, time and channel, and at the designated time, the system either can switch to that channel and allow you to view the program or record the program for later viewing. After changing channels to watch Shakira do her thing for a while, I decided to check out the M20's streaming audio feature. Streaming audio works about like you'd expect and sounds as good as most of the small radios people bring to work. But, that's where the three RCA plugs (both audio and video) on the side of the unit come into play. The M20 can be connected to standard AV equipment allowing you to take advantage of the bigger screen and better (amplified) speakers you probably already own. I easily could see using this unit, connected to The Big Amp in the living room, to play streaming music at parties or while we worked around the house. In my experience, Internet radio stations tend to play better music than over-the-air radio stations, so this would be a great thing to have at the home or office. Any time I see a device with a USB port on it, it piques my interest, and the M20 was no exception. Once I plugged a pendrive in to the port, located on the back of the device, I was able to play any MP3, AVI or MPEG file on the drive as though it were a streaming media www.linuxjournal.com October 2007 | 49 FEATURE Tornado M20 source. I was a little dismayed to discover that plugging in a USB key¬ board would crash the phone. USB keyboard support might have been nice for some of the messaging features discussed later. The Owner's Manual indicates that the phone can scan the network for open shares that contain music and video files. I wasn't able to test that feature in time for this article, but you can bet I will soon, and the configuration options indicate that this should be fairly easy to do. I really like reading news via RSS. Usually, you get a lot of diverse headlines and just enough summary information to let you determine whether the article is interesting enough to read in depth. However, this function on the M20 points out its two biggest weaknesses. First, the font is small and sometimes difficult to read. It is readable, but it takes some time to get used to it. But, the biggest problem comes from the fact that if you see a news headline you want to read more fully, you can't, because the M20 doesn't have a Web browser. Now, I understand that it's asking a lot to embed a Web browser in a tele¬ phone, but the newsreader feature just screams for it. The M20 has a weather function that displays the local weather forecast for the next four days. Having gone through all of the config¬ uration options, I'm not sure how this function works. I suspect that the weather information is pushed to the phone from a central server. Other minor services worth mentioning include a stock tracker, currency converter and, yes, as I noted earlier, a calculator. The M20 also has e-mail, instant messaging and SMS messaging features. I wasn't able to test the e-mail capability directly, as the unit sup¬ ports only the POP protocol, and my Courier POP server didn't want to October 16-18,2007 San Jose McEnery Convention Center San Jose, CA * www.ispcon.com ISPCON is where the service provider industry goes to GET REAL about the future of their businesses. Isn't it time you increase ARPU? Pump up profits? Optimize your operations? Find out what's next? Whatever your plans, ISPCON will help you GET REAL about them. This is the industry's only forum where peers learn from peers in real one-on-one discussion about what works, what doesn't and what's next. GET REAL! Get to ISPCON! Register online and use Customer Code LNXF7 to get your cooperate. However, explaining this feature is fairly easy. Once properly configured, the M20 can be used to check a user's e-mail quickly. The M20 doesn't support sending or replying to e-mail, only reading it. Now, at first, this may not seem like a very useful feature, but imagine rushing into your office on your way to an 8:00am meeting and using your telephone to skim your incoming e-mail quickly to check for any important messages. Now, imagine being able to do this in less time than it takes your PC even to boot up. How I wish the unit supported IMAP, but it would be worthwhile for me to configure a POP server just to use this feature. Because the M20 lacks a keyboard, the IM function seems like it would be a bit cumbersome. But, as the unit supports ICQ, MSN, AOL and SMS messaging, it's very much worth mentioning. The initial configuration of the IM client is done via a Web browser. Once configured, you can send and receive instant messages on any of the supported networks. Sending a message involves using the phone keyboard to type messages, just like you would to send an SMS message on a cell phone. So, as you can see, the M20 packs a lot of features in one little footprint. But, how well is it supported? As part of this evaluation, I had intended to contrive a fictional, yet plausible, problem with the unit and call SysMaster's tech-support line. Fortunately for me, the unit arrived misconfigured, and I had a perfect opportunity to visit with one of SysMaster's support technicians. I already had determined that the SIP password had been miscon¬ figured, but I let the technician guide me through resolving the problem, which he did in a logical progression—all the while exhorting various features of the device that he thought I may not have been aware of. He obviously knew the equipment and the technology. I was relieved not to be talking to someone who was simply reading a script. As you might imagine with a device as sophisticated as this, configuration is everything. Sadly, very little of this configuration is exposed via the built-in video screen. Almost all of the configuration is done with a Web browser via HTTP. Because the Web-based adminis¬ tration console didn't work with Konqueror, I had to use my second choice, Mozilla Firefox, which worked well. At least I didn't have to resort to some other buggy browser. The administration console wasn't as straightforward as I would have wished. There is a lot of configura¬ bility in this device, but in my opinion, it isn't well organized. So, despite a few quirks here and there, this is an amazing device. The M20 brings video, voice and text communications together in one convenient platform. Wireless, as well as LAN connectivity, plus its adherence to open standards, allows the M20 to be deployed just about anywhere. ■ Mike Diehl works for Orion International at Sandia National Laboratories in Albuquerque, New Mexico, as a Linux Nerd. Mike lives with his wife and two small boys and can be reached via e-mail at mdiehl@diehlnet.com. Resources SysMaster: www.sysmaster.com The M20 IP Video Phone: www.sysmaster.com/products/video_phone.php "Building a Digital Lifestyle with Open-Source Technology" by Mike Diehl, LJ, October 2006 (about my MythTV and VoIP configuration at home): www.linuxjournal.com/article/9111 50 | October 2007 www.linuxjournal.com Growing a World of Linux Professionals LPI-Deutsch LPI-Bulgaria LPI-South Asia LPI-Latin America LPI-South Africa LPI-Korea LPI-Japan LPI-China We at the Linux Professional Institute believe the best way to spread the adoption of Linux and Open Source software is to grow a world wide supply of talented, qualified and accredited IT professionals. We realize the importance of providing a global standard of measurement. To assist in this effort, we are launching a Regional Enablement Initiative to ensure we understand, nurture and support the needs of the enterprise, govern¬ ments, educational institutions and individual contributors around the globe. We can only achieve this through a network of local "on the ground" partner organizations. Partners who know the sector and understand the needs of the IT work force. Through this active policy of Regional Enablement we are seeking local partners and assisting them in their efforts to promote Linux and Open Source professionalism. We encourage you to contact our new regional partners listed above. Together we are growing a world of Linux Professionals. Linux Professional Institute Stable. Innovative. Growing OpenMedia myPVR 2.0 A nicely polished Myth configuration on excellent hardware with slightly weak HD performance. JES HALL MythTV is arguably the most popular multimedia platform for Linux. It has excellent support for television—allowing you to pause, fast-forward and rewind live TV streams by spooling the stream temporarily on disk. It has commercial detection, so you can skip advertising, and it can use a variety of tuners and video input devices, including DVB- and ATSC-based digital tuners, as well as traditional analog tuners both with and without hardware MPEG-2 encoders. It also includes a music player interface capable of ripping, encoding, sorting and playing most popular audio formats, with nifty full-screen visualisations. It will rip and transcode DVDs and play almost any video format you can imagine off the hard disk. One of the weaknesses of MythTV is that it's not very easy to set up and configure; it's hours of hard work for people who know what they're doing— vastly beyond the capabilities of average consumers. myPVR, from OpenMedia (openmedia.co.nz/openmedia/content/ section/3/44) provides a PVR solution built on MythTV that smooths over most of the rough edges. OpenMedia has put a lot of energy into polishing the MythTV interface, and it shows. When requesting a unit to test, we were asked what kind of display we'd be using, so that the profiles could be set up for us. When it arrived, it worked straight out of the box, without any config¬ uration required, and all of the PVR's functionality could be accessed using the remote control. The New Zealand TV channel listings already were set up for us when we turned on the device. 52 | October 2007 www.linuxjournal.com WilriiTY Uc5atlbr|vy . Uj-’JSH Itaaidirrjs /m liConnii'in Ce'tsi m Op*.til CuH V T 1 myPVR w Figure 1. myPVR Default Theme II Time from box arriving to media playing: two minutes. II Time from box arriving to first crash: ten minutes. W J In order to stress-test the system, we played BBC's Planet Earth in 1080p H.264. It tried gamely, but we managed to crash the media player in fairly short form. This is excusable though, as almost nothing manages to play H.264 content well on Linux. It did, however, perform very well on 720p HD content and SD content. The sound card was very well isolated, with crisp sound that had almost no discernable hiss, even when we turned up the amp as high as it would go. The box itself is a little noisy—if you were to have it sitting across the room, it probably wouldn't be noticeable, but behind the couch in our Official Audio/Visual Testing Suite (er, Jes' living room), the fan noise was audible during very quiet scenes. Still, we were very pleased with the choice of hardware. Some other nice touches are a slick default theme, although it is a little like Windows Media Center. The New Zealand channel guides were programmed for us already, as well as little touches like the official channel icons. The unit itself looks very appealing, with a slick black case and muted-blue glowing LCD showing the internal temperature and fan speed. Some fun games were pre-installed, including Frozen Bubble , Tux Racer and the XMAME emulator. The specifications claim support for USB gamepad and joystick devices, but unfortunately, we couldn't get it to work. We feel this is probably a make-or- break issue for gaming on the device—if USB gamepads worked without requiring arcane configuration, we imagine it would be quite popular for casual gamers who were not interested in buying a dedicated console. Another excellent improvement from the default MythTV distribution is the ability to download and apply updates to the software from OpenMedia, using a reasonably simple update interface and a tool to configure net¬ working easily. y Th* MeiinCirttn Fhhln’ r/M *«*™ vp*™ vw™ vi* i Figure 2. myPVR TV Interface Steven Ellis from OpenMedia was kind enough to talk to us about MythTV, his product and DRM. JH: So, how did you get started in MythTV? SE: I played with video codecs back in the UK as a developer, so I've always been into that side of things. I kept an eye on Freevo and MythTV and did test installs, but I never felt the hardware was good enough for what I was after. Plus, one of the guys at my old job was playing around with TiVo and MythTV, which gave me some exposure. Then, about 18 months ago, I felt there was a good-enough hardware platform for my needs, which eventually became myPVR 1.0. JH: What were the issues with the previous hardware platforms? SE: Speed wasn't there to do HD, or you had to do a lot of cooling. I was after as integrated a solution as possible, and it had to be future-proof with good composite/svideo TV out as well as DVI/HDMI support. The NVIDIA 6150-based platform was perfect—great graphics with good drivers, excellent TV out and all integrated into the MB. JH: How do you feel about MythTV as a platform—how far it's come, and where it's likely to go? SE: The video side of it rocks, but for music, it has a long, long way to go. OpenMedia myPVR 2.0 $1,999 NZD ($1,565 US) HARDWARE: II ASUS/nForce Socket AM2 Athlon 64 motherboard. II AMD Athlon 64 3600 X2 CPU. II Dual-layer DVD+RW with DVD-RAM. II 512MB DDR2 SD-RAM. II Gigabit Ethernet. II High-definition audio with analog and digital outputs. II NVIDIA 6150 integrated HDTV-capable video. II 250GB SATA hard disk. II FireWire, front and rear. II Six USB 2.0 ports. II Hauppauge MPEG-2 video capture card. II Remote control. We tested with a Panasonic PT-AE900u 720p projector and a standard 29" PAL television set. People who have used other PVR software solutions are always blown away by how quick MythTV is, but they are shocked at the MythMusic plugin. It was okay three years ago, but it doesn't measure up against iTunes. As a platform, it provides an amazing base, but it is still hard for the average user to customise, which is why OpenMedia provides myPVR fully configured. As to where it is going, lots of interesting work in the digital TV space is where we are playing catchup to MediaPortal. Also, there's some nice work in the IPTV space, but we always will be hindered by the proprietary DRM solutions on the market. Commercially, the biggest gap right now is the lack of Blu-ray and HD-DVD support. I have to be www.linuxjournal.com October 2007 | 53 FEATURE OpenMedia myPVR 2.0 very careful about any statements I say, as I don't want potential customers assuming support. JH: Do you think this is going to be a big issue for MythTV and myPVR? SE: We have to keep up with the technologies. We showed off myPVR at Auckland's Big Boys Toys show last year, and the feedback was awesome, but a lot of consumers assumed it would already have HD-DVD or Blu-ray. Geeks usually can get by with the technology at hand. I think that DRM and potential legal issues are a bigger risk to MythTV, and myPVR. The pending changes to the New Zealand Copyright bill will make myPVR effectively illegal. JH: What changes are those? SE: The law finally makes time shifting legal, which is great, but some of the rules and provisions are simply idiotic. For example, you can time shift only material that isn't available "on demand". How are you supposed to police that? It also provides a lot more weight to the providers of DRM. They capture most of this under TPM (Technological Protection Methods). It becomes illegal to remove, tamper, reverse engineer or study TPM except for a couple of exceptions. Where it really hurts is that it becomes illegal to provide, produce or sell equipment or software capable of removing TPM. That hits us hard, as DVD playback on Linux is effectively TPM removal. JH: Yeah, that's pretty rough. SE: Plus, we might be required to enforce the restrictions on time shift¬ ing. For example, our units would not be allowed to keep material for more than a reasonable amount of time, but there is no definition of what a reasonable amount of time really is. Days, weeks, months? There also is a provision for format shifting of audio, but not video or pictures. Hence, we shouldn't allow our cus¬ tomers to copy their old VHS tapes to HD or DVD, as that is breaking the law. They are pushing the policing to the equipment providers. Overall DRM is a potential killer for nonproprietary devices like MythTV. JH: I'm assuming this is the same in the US as well? SE: Consumers would love a one- device-fits-all product, but with the advent of competing DRM systems for IPTV on top of the DRM used by traditional pay TV companies, there is a lot of lockout. At least some of the US cable companies provide cable boxes that do the DRM that have FireWire connectivity for PVRs to hook into. Then, there is the DRM in the audio space. Consumers just get annoyed when their iTunes tracks don't play on their Zune, etc. JH: It all sounds pretty bleak really for Linux PVR devices. SE: All PVR devices. Microsoft has similar issues with Media Center. But, It also includes a music player interface capable of ripping, encoding, sorting and playing most popular audio formats, with nifty full-screen visualisations. as shown in the UK and parts of Europe, there is a large market for FTA (Free to Air) services. We are finally starting to transition to FTA digital here. Thankfully, New Zealand is going for a third-generation digital terrestrial system based off H.264 rather than the bandwidth-hungry MPEG-2 HD you get in Australia and the US. One of the greatest strengths of MythTV is the global development. The support for the HD tech we will use here already has been written and tested by guys in the UK. JH: Are there any good open-source alternatives to MythTV? SE:MediaPortal is open source but on Windows. Some people love Freevo, but it is more of a wrapper around existing tools, rather than an integrated solution. The Elisa work from the Fluendo guys also is really interesting (elisa.fluendo.com). At the moment, their focus has been on playback, but when they have the PVR features, it could be a real killer app. JH: So, what else would you like to share about myPVR before we wrap this up? SE: A couple odds and sods.... We try to contribute back where we can—for example, support for the New Zealand MHEG profile for digital interactive TV in both MythTV and the external RedButton application. All the work has gone back to the community. It's not a lot of code, but there was a hell of a lot of testing. Plus, we had full support the day the Freeview network launched in New Zealand. Another area we got involved in by accident was a user interface for a customer with cerebral palsy (www.mythtv.co.nz/ mythtv/remote/remote-wide.html). He can't use a normal remote control, but has a special keyboard/mouse. With this Web remote, he can use all of the features of his PVR. JH: That's a very cool example of the flexibility of open-source software. SE: Yeah. He helped iron out the niggles and just loves the way it works. Finally, we were originally based on KnoppMyth, but we currently are col¬ laborating with the Mythbuntu team to get some of the tricks we've learned included in their build. JH: Thanks for your time. myPVR 2.0 is a pretty polished product given the limitations of the platform. It doesn't diverge far from standard MythTV, but what it does do is put together a solid product that saves you many hours of pain attempting to configure it yourself. We give it a thumbs-up and look forward to seeing how it improves with Mythbuntu.■ Jes Hall is a UNIX systems consultant and KDE developer from New Zealand. She’s passionate about helping open-source software bring life-changing information and tools to those who would otherwise not have them. 54 | October 2007 www.linuxjournal.com 23-24 October 2007 • Olympia 2 • London Open for business LinuxWorld Conference & Expo2007 is the premier event focusing exclusively on Open Source and Linux technologies. The show, which attracts technical decision makers and business professionals will showcase the latest technology, debate the use of Open Source in the desktop environment, touch on the use of Linux in the datacentre and discuss the development of a common Linux-based platform for mobile phones. Attend the groundbreaking conference, where you can see world leading technical experts discuss the industry's more relevant topics* Aimed at technical and business professionals, the 2007 conference will offers: • 2 full days of content, covering many fields • 2 conference/masterclass streams • Indepth technical sessions • Business focused seminars As well as being able to visit over 60 exhibitors, showcasing new products and services, the Expo offers wide reaching benefits to you, the visitor: • On the floor business briefings, showcasing real life examples of advantages, applications and integration of Linux and Open Source in the business • Great Linux Debate • Take LPI Examinations at a special visitor discount • Meet developers and technical experts in the *org Village WWW* linuxworldexpo co*uk WWW* linuxworldexpo co,uk • IDG WORLD EXPO 2007 IDG Word Expo Corp. ALL rights reserved. LinuxWorld and LinuxWorld Conference & Expo are registered trademarks of International Data Group Inc. All other trademarks contained herein are the property of the respective owners. All trademarks acknowledged. E&OE. Programme may be subject to change. Correct at time of press. Getting Started with the Trolltech Greenphone SDK Everything you need to know to start programming for the cool new Greenphone. ROBERT E. HARTLEY 56 | October 2007 www.linuxjournal.com T rolltech recently released many smartphone developers' dream combination—the Linux-based Greenphone and its open-source Qtopia Phone SDK. The Trolltech Greenphone is a full-featured tri-band GSM (900/1800/1900MHz) mobile phone with a built-in 1.3 megapixel camera. Like many other modern smartphones, it features a QVGA touchscreen, Bluetooth, client USB, mini-SD Flash and stereo audio connectors. Under the hood, it is built around a Marvell 312MHz PXA270(ARM) processor, runs with 64M of RAM, and has 128M of built-in Flash storage. It also uses the same field-proven Broadcom BCM2121 GSM/GPRS baseband processor module as the Palm Treo. The Greenphone almost could be described as a sporty, Linux version of the Palm Treo. It is much lighter, more compact and comes in any color you want—as long as it's green! Introducing the Qtopia Greenphone SDK The Greenphone SDK is a distribution of Qtopia Phone Edition, specially configured for use on the Greenphone. Qtopia (formerly Qt Embedded) inherits its API from Trolltech's flagship product Qt, a mature C++ application framework available for Linux/X11, Windows and Mac OS X. Qtopia is specifically enhanced for embedded Linux devices, such as phones, PDAs and appliances. Qtopia is remarkably self-sufficient—sitting immediately on top of the base operating system, it provides its own font and image render¬ ing, window management and input processing. Along with this sort of behind-the-scenes stuff, Qtopia avails itself of all the rich features of Qt's standardized access to files, networking, multithreading, interna¬ tionalization, graphics processing and, of course, GUI widgets. Like its big brother, Qtopia is readily available either as a free download to those wanting to do GPL-style open-source development or by commercial license purchased from Trolltech. Qtopia also bundles a suite of personal information management tools as would be expected on any PDA or smartphone. Inside the Box When you open the Greenphone's box, you find everything needed to start developing: mobile handset, power supply, USB cable, stereo headset and a Greenphone SDK CD-ROM. This last item contains a copy of the VMware player and a virtual machine preloaded with Debian Linux, Qt, Qtopia, KDevelop, native development tools and a complete ARM cross-compiling toolchain. www.linuxjournal.com October 2007 | 57 FEATURE Trolltech Greenphone SDK Gnokii For an excellent example of a Linux program using GSM AT command codes, check out Gnokii—it's a great tool for learning about GSM modems, as you actually can watch each transaction with the data sent and received to exe¬ cute commands to dial the phone, send SMS messages and so on. It works well with most modern phones that have a serial or USB data connector. By the time this article is printed, it might already be running natively on the Greenphone. For complete details, visit www.gnokii.org. The only other item needed actually to go on air with your newly created applications is a GSM subscriber identity module (SIM) card with a currently active account. What Is GSM? The two most widely used standards for cellular communications are GSM and CDMA, with GSM outnumbering CDMA worldwide by a ratio of about 1 billion to 270 million users. Although they are as completely incompatible on each other's networks as AM and FM radio, a normal user typically would notice only that phone accounts on GSM phones are usually tied to the SIM card, so changing handsets is really easy. GSM currently stands for Global System for Mobile communica¬ tions. It is an open standard governed by the nonprofit European Telecommunications Standards Institute (ETSI). GSM allows great interoperability permitting easier international roaming. GSM's other claims to fame are higher digital voice quality; a cheap short messaging service (SMS); multimedia messaging service (MMS) for exchanging images, audio, video and rich-text data; and finally, a general packet radio service (GPRS), often used as an Internet gateway, with billing based on megabytes of data transferred, as opposed to time spent on-line. The Greenphone actually is based on two ARM RISC processors: a beefy Marvell 312MHz PXA270 acts as the PDA part and communi¬ cates to the ARM7-based GSM/GPRS communications controller. This controller is effectively the heart and brain of the phone component. It also acts like a modem and accepts GSM AT-style commands to dial, answer calls, send or receive messages, and other activities typical of a GSM/GPRS communications device. Standard GSM modem AT commands can be sent to the phone, which execute and then return a response code. These AT commands are analogous to the decades-old Hayes modem commands, but instead of using or programming a string like "+++ATDT16505511676" to dial Trolltech's number 1 -(650) 551-1676, we do the same sort of thing with the GSM version of the command set, and the string looks more like "ATD16505511676". There are myriad other AT commands for handling everything that the GSM/GPRS phone unit can do. These include setting up the phone and establishing various types of communication, such as voice calls, SMS and MMS, and they can be used for getting information on signal strength, network status and so forth. For a comprehensive description of these standardized commands, you can download the following documents from the ETSI site: 1> ETSI TS 127 007 V3.13.0 (2003-03), AT Command Set for 3G User Equipment (UE): webapp.etsi.org/exchangefolder/ ts_127007v031300p.pdf II ETSI TS 127 005 V7.0.0 (2007-03), Data Circuit Terminating Equipment (DTE-DCE) Interface for Short Message Service (SMS) and Cell Broadcast Service (CBS): webapp.etsi.org/ exchangefolder/ts_127005v070000p.pdf Qtopia embeds the most commonly used of these commands into C++ class wrappers. The on-line documentation in Qt Assistant describes how to use and enhance these classes in the sections titled "GSM Modem Integration" and "Modem Emulator". For testing applications off-line, the Greenphone SDK provides a modem emulator that allows testing software without having actually to join a network and go on air. Getting Ready Loading the development environment is simple—run the installation program on Windows or Linux, respond affirmatively to the prompts, and within a minute you will have VMware and the SDK tools, applica¬ tion sources, documentation and binaries installed with an icon on your desktop to start things up. This makes life really easy for reluctant developers using MS Windows to get into both embedded and desktop Linux and Qt application development. At the time of this writing, developers using the x86 version of Macintosh OS X can use the Greenphone SDK under VMware Fusion, but they need to copy over the virtual machine's files from another installation; however, this may change by the time this article is published. Getting Started Building Applications Trolltech always ships its products with copious documentation and example code demonstrating all common features, and the Greenphone SDK is no exception. For starters, the "Developer Quickstart Guide" shows what needs to be done to build an application with a few one-liners. First, we start the Qtopia emulator using the Qt Virtual Frame Buffer and a Greenphone skin by clicking on the runqvfb icon on 11TIP: One technique for VM-based cross-platform development is to export your display from the Linux VM to your host machine running an X11 server. This might be the built-in X11 server running locally on your Linux host, Apple's optional add-on to OS X or even Cygwin on a Windows machine. I use screens rotated 90° to allow reading many more lines of code without scrolling, so this trick helps to leave the VM configurations as generic as possible. GUI performance typically is enhanced when the X server is run on the host machine due to the lowest level rendering being shoved off as far down the pipeline as possible— often at the display adapter's GPU. Using this method, it can almost make Windows and OS X feel like a Linux box. 58 | October 2007 www.linuxjournal.com our desktop. This is analogous to an X server for Qtopia, and it provides an exact pixel-for-pixel representation of the program running on the phone. Then, we start the Qtopia phone environment by clicking the runqpe icon, which then connects to the qvfb process and displays its contents in its virtual screen. Figure 2. qvfb Running with qpe Running in It We need to run a script to set our QPEVER and PATH environment variables and to define some functions for communicating to the phone. If building for the x86 version of Qtopia, we would use: . /opt/Qtopia/SDK/scripts/devel-x86.sh Otherwise, if building for the actual Greenphone itself, we would choose the cross-compile environment with: . /opt/Qtopia/SDK/scripts/devel-greenphone.sh Then, we change to our directories and build: cd -/projects/application qtopiamake -project && qtopiamake && make && gph -p -i The qtopiamake program is Qtopia's version of the Qt qmake utility. It can generate a .PRO project file based on the contents of the current directory if given the -project parameter, but its most important job is to use the project file as the starting point to generate a Makefile based on the installed configuration of Qtopia and the type of build we want. Typically, we generate a new .PRO and Makefile only when we have new files to add to our project, but qtopiamake takes so little )] NOTE: It might be worthwhile to point out that the commands depicted here are separated by a double ampersand (&&) to cause execution of the command string to stop at the first point where it meets an error. In this case, it would stop the shell from trying to execute or install a program that had failed to build. gph Utility The gph utility is a useful awk script that provides convenience functions for doing things like building, packaging, installing and running applications in the Qtopia environment, either emulated or actually on the phone. It can be invoked instead of make, and it is powerful enough that we can use it to convert from raw source code to running applications installed from a package onto a real or emulated device. For example, to rebuild the code, package it and install and run on qvfb, we would try the following command: gph -C x86 -debug -c -all If that were to go well, and we were happy with enough to want to run it on an actual phone, we could build ourselves a package to install with the following: gph -C greenphone -p This would give us a file in our pkg directory with a name like qpe-application_1.0.0-1_arm.qpk. To have a more complete idea of the gph capabilities, we can look at its help message: Sgph --help FORMAT: /opt/Qtopia/SDK/scripts/gph Options: -h display command help -q quiet mode -c clean -C reconfig or , default greenphone -b make -p make package -i install package to greenphone -r run application on greenphone -all reconfig, make, make package, install, run -debug force configure for debug build, override default -release force configure for release build, override default -options "" pass extra options to build system eg. /opt/Qtopia/SDK/scripts/gph -C x86 -all -options "CONFIG+=debug" -net bring up network to greenphone -console console on greenphone -qpk install qpk to SDK http server for installation -rescan Tell Qtopia to rescan for new documents and applications w www.linuxjournal.com October 2007 | 59 FEATURE Trolltech Greenphone SDK )] NOTE: As a brief reminder to newer users of Qt, classes generated by Designer do not have a base widget that contains all the other widgets inside it. The code in the generated class instantiates child widgets of only whatever parent widget instance is passed into its constructor. This is a reason why we see multiple inheritance used with classes derived from Qt Designer-generated code—it provides a single widget from which to hang other widgets. time to execute that it is common to see it run from a standard shell script every time. When applications are packaged for distribution to Qtopia devices, they reside in .qpk files that are specially made gzip'd tar files. In addition to the executable file itself, there also are the help files, multilingual translation files and the desktop entry. This desktop entry file contains elements to describe the name of the executable, the icon, what type of application it is and, finally, the MIME type to indicate what sort of data it can process. Debugging on Target Debugging our application on the Greenphone itself is fairly easy to set up. First, we need to configure our build environment to cross-compile for the ARM architecture on the Greenphone: . /opt/Qtopia/SDK/scripts/devel-greenphone.sh Next, we need to rebuild our Makefile and executables, package them up and install them onto the phone: qtopiamake -debug && make clean && make && gph -p -i -r -debug At this point, we can click the arm-linux debugger icon on the desktop, open the example program on the phone and click run to execute it in the remote debugger. Building for Development and Deployment A number of application deployment methodologies are available when developing with the Greenphone SDK. First, we can build and run our applications so that they execute within the virtual machine's ersatz phone-like environment complete with a JPEG skin that looks like a phone sitting on your desktop. It is great for working out the GUI parts of an application, and it saves us the step of having to compile, flash on to the unit and debug remotely. Second, applications can be compiled and packaged for the Greenphone, then run and tested on the device itself. As mentioned previously, the VM installs with a gdb debugger that can run inside the virtual machine and debug applications remotely on the phone. Finally, we have "Phone bounce" mode, using the phone as a standard GSM/GPRS modem connected to the development machine via a USB cable. Our applications would run inside the emulator on the local machine, but would be joined logically to the real phone by whatever network link you have to it, such as a USB connection. We do not have to customize the Greenphone in a piecemeal fashion. The Flash image representing the whole Greenphone Linux distribution on the phone can be updated using either the mini-SD Flash reader or the USB cable to re-install a complete new Linux distribution along with all the applications and configuration files. This makes it easier for enterprise use where items such as stan¬ dard configuration options are set to enable things like corporate branding with visual themes, VPN settings, default address-book entries and localization settings, among others. Anatomy of a Qtopia Application Qtopia development will make any Qt or KDE developer feel right at home, as it is quite compatible with the desktop version of Qt. There are a few minor differences, as we will see in the example application in the Greenphone SDK found at -/projects/application. The style lately, with C++ in general and Qt v4.x in particular, is to include a header named after the class we want declared. This saves ever having to guess which header contains a class' declaration. In the following example, we have the old way commented out and the easier-to-remember method following it: // main.cpp #include "example.h" // #include #include QTOPIA_ADD_APPLICATION("example", Example) QTOPIA_MAIN // end of main.cpp The function formerly known as main() has been deprecated in Qtopia in favor of two macros. In the above example, the QTOPIA_ADD_APPLICATION macro is used to create an instance of the main application window. The first parameter is the executable name, and the second parameter is the base class of the application window class. The QTOPIA_MAIN macro expands out either to the traditional main() function if building a traditional application or to the entry point needed if building a quick launcher plugin. Inside our example.h, we find the class declaration for our main window, which we have sub-classed from a generic QWidget: #i fndef EXAMPLE_H #define EXAMPLE_H #include "ui_examplebase.h" class ExampleBase : public QWidget, public Ui_ExampleBase { public: ExampleBase( QWidget ^parent = 0, Qt::WFlags f = 0 ); virtual ~ExampleBase(); }; class Example : public ExampleBase { Q_0BJ ECT public: Example( QWidget ^parent = 0, Qt::WFlags f = 0 ); virtual -ExampleQ; private slots: 60 | October 2007 www.linuxjournal.com void goodBye(); }; #endif // EXAMPLE_H void Example::goodBye() { closeQ ; } This class uses a form created using the Qt Designer GUI building tool, so we see an include file called ui_examplebase.h that brings in its declaration. In Qt, headers with names starting with ui_ typically are Designer-generated. This is followed by our class' immediate ancestor called ExampleBase. This base class inherits from both QWidget and the class defined by the GUI builder called Ui_ExampleBase. Figure 3. Designer Running in VM Our main window is an instance of the Example class derived from ExampleBase. It makes use of a technique called signals and slots—a method used by Trolltech that allows great flexibility for defining how a function is invoked. The invoking side of the connection is called a SIGNALO, and the invokee side is called a SLOT(). They are joined together using a method called connect() that allows a many-to-many connection relationship. Qt uses a preprocessor to add metadata processing to add to C++ dynamic invocation and object introspection effectively and elegantly—elements available in other OOP languages. Our final code example shows the implementation of our classes: #include "example.h" #include ExampleBase::ExampleBase( QWidget ^parent, Qt::WFlags f ) : QWidget( parent, f ) { setupUi( this ); } ExampleBase::~ExampleBase() { } Example::Example( QWidget *parent, Qt::WFlags f ) : ExampleBase( parent, f ) { connect(qui t, SIGNAL(clicked()), this, SLOT(goodByeQ)); } Example::~Example() { } Our ExampleBase class' constructor calls the Designer-generated setupUiO method to have the form-defined child widgets created and their layout and other properties set. Without that step, it would be a generic QWidget. The next interesting thing we see is the constructor for the Example class. It calls the connect() method to join the clicked() signal on the Qt Designer-generated QPushButton called quit with our goodBye() slot. This allows us to exit the example program by clicking the QPushButton labeled Quit. Where to Go from Here It would be great to explore many other Qtopia-specific features in more detail, such as the QCop IPC mechanism for communication across processes, the XML-based theming engine, the secure execution environment (SXE), plus all the phone-specific features, for starters. Qtopia provides a lot of advantages for the embedded developer designing applications for mobile phones. Besides the emulation environment, the code compatibility with desktop applications and the natural intuitive application framework, I find that Qtopia does something better than other embedded environments do—it helps put the fun back into programming. ■ Robert E. Hartley is happily married and the father of four wonderful children. He is a dual-national Australian/Canadian living in Ottawa, Canada. Robert started his working life as a programmer writing assembly language on IBM mainframes before getting hooked on UNIX and C in the mid-1980s. He started using Linux as his principal desktop environment ten years ago, and has until recently been busy consulting and training customers around North America and Europe as a Trolltech certified Qt Trainer for its North American training partner, ICS (www.ics.com). Resources An introductory comparison between CDMA and GSM for the North American market: www.wisegeek.com/ what-is-the-difference-between-gsm-and-cdma.htm European Telecommunications Standards Institute (ETSI): www.etsi.org GSM: en.wikipedia.org/wiki/GSM "Using Qt to Develop for Embedded Linux" by Natalie Watson (. LJ , May 2001): www.linuxjournal.com/article/4660 "Choosing a GUI Library for Your Embedded Device" by Martin Hansen (LJ, January 2007): www.linuxjournal.com/article/9403 Short Message Service: en.wikipedia.org/wiki/Short_message_service Multimedia Messaging Service: en.wikipedia.org/wiki/Multimedia_Messaging_Service General Packet Radio Service: en.wikipedia.org/wiki/General_Packet_Radio_Service www.linuxjournal.com October 2007 | 61 Is a Promising Work in Progress KDENLIVE is the Kracker-Jack video editor with the Kantankerous name. Dan Sawyer file fdir ^iew Emjer.r Imn^line Mnnirnr Render I aycur Windows ^errintis hjelp V -V Ob ' ui lh H ® lirUl J l rt 12 3 4 & KDENLIVE I-Vb-I MiujviL.hdenl^ - NTSC [rnudilieJJ - Kdenlive Project Tree Transition Fffect I ist Fffer.r Stack E> Search: Thumbnail ritenarriG SS£ Description "Tn -3 i *i datyavici™.dv kayl.mov kGyP.mav lachJnspecLiun.rnpey lachreel-lfiKQ.vflb wcilt.niov 43 1 (.1J Timeline Monitor £lip Monitor Capture Monitor Wriiiha IIWBI./ DIFFERFNnr 0:00:40.00 0:00:^0.00 0:01:00.00 0:01^10.00 ,1 0:01:20.00 0:01:30.00 0:01:40.DU 0:01 :b0 0 D 4 .kinofx.ctv Q 04 .kinofx .ctv 004.kinQfx.dv i :niffc ip . keyl.mov r Tracks Scroll Lf | Ready. a < k Mnve)Resl7e mode NTSC (7?flx4BD, TlOfps) Figure 1. Manipulating the Project Tree in KDENLIVE I had a stage play to edit. I didn't shoot it, and that lack of quality control on the head end meant that the editing was not the straightforward matter it should have been. It was shot in HDV, which is stored as an MPEG-2 transport stream, and my normal editing program doesn't play nicely with MPEG-2. Because it was a rushed job, I didn't have time to pull in a developer to add HDV support to it. So, I was faced with the task of turning around a difficult edit on a short timetable, using footage in a file format my favorite editor couldn't read. Fortunately, this is open source. And, the odds are that somewhere, someone has run into the same problem before and done something about it. In this case, 62 | October 2007 www.linuxjournal.com my friend Dan Dennedy (developer of Kino) pointed me to a new video editing project named KDENLIVE (yes, it really is an acronym: the K Desktop Environment Non-Linear Video Editor), which is based upon his multitrack engine MLT (Mutton, Lettuce and Tomato—the XML file format it writes to, appropriately enough, is called Westley). I had run across KDENLIVE before, but had run away just as fast. Yes, Virginia, there is a hell. And, it's a very special hell, more heinous than that reserved for child molesters and people who talk at the theater, grander than Dante's ninth circle reserved for the Borgia dynasty of Popes, with more exquisite torments. This tenth circle, invented in the digital age, exceeding any that came before it, is the bane of Linux users everywhere, Dependency Hell! This special torture is reserved especially (though not exclusively) for those of us intent on bending the mighty Linux dragon to our will, forcing it to perform feats of derring-do in the creation of art. Occasionally, in this never-ending struggle to do with open source what people with bigger wallets do easily on a Mac, the erstwhile adventurer stumbles upon a tool, just on the point of maturity, suitable to be appropriated to the task at hand. This time, I didn't have the option to run away from KDENLIVE. Blender wouldn't do— the Video Sequence Editor doesn't perform well enough on my machine to edit HD with real-time playback. Cinelerra wouldn't do—a program that crashes every few mouse clicks isn't my idea of fun. I could have done it in Kino, but that would make for nightmares conforming the audio later, as I'd have to patch it up manually in Audacity when the video cuts had to jump in the middle of a word (not to mention the necessity of con¬ verting to DV beforehand, which meant chewing up another 200GB of hard-drive space I wasn't keen on parting with). None of the other projects on the radar looked like they offered any significant advantage over KDENLIVE, so I bit the bullet and downloaded the thing, steeled with a vial of holy water for my descent into that very special Dependency Hell. I started working my way through the list and...everything worked. The install was almost flawless, aside from a couple tweaks that had to be made to the MLT build parameters to accommodate a 64-bit environment. Within about an hour (and that spent building the extensive list of dependencies from source), I was up Stacking up against previous open-source forays into multitrack editing, KDENLIVE shows definite areas of improvement. and running. Well, running might be overstating it a bit. This was KDENLIVE version 0.4, and it was dog slow. To bisect a clip, the work flow went like this: 1. Select razor tool. 2. Choose cut point. 3. Click on the cut point to make the cut. 4. Go into the kitchen, pour a fresh glass of iced tea. 5. Come back 45 seconds later just in time to see the cut update to the interface, and continue working. No, I'm not exaggerating for comic effect. This clearly was not going to work. I couldn't bill my client for time spent on a laggy interface, and I didn't much relish the thought of spending several orders of magnitude more time on the project than I could honestly bill for. As a last-ditch effort before borrowing a Final Cut HD machine for the weekend, I gave the KDENLIVE SVN version a try. This one worked. It was still a little slow, but it was certainly serviceable. It played the HDV back in real time, and its lack of an HD edit profile, while annoying, was not a deal-killer as my client wanted a standard-definition DVD as the end product. A Multitrack Editor KDENLIVE presents itself pretty much as a standard multitrack editor—its interface looks similar to versions of MainActor and Premier from years past. All visible and within easy view in nested, dockable tabs are as follows: II An asset management window. II Effects control dialog with the standard crop of basic transitions. II Clip selection, timeline and video capture viewer ports. II A multitrack a/v timeline supporting infinite tracks. Stacking up against previous open- source forays into multitrack editing, Krtrm ivr i*%i pi i i Pe EtS. froiM Irfwlnt MCrttw Ftenflfr Lays*/ SfClinaa He** * j v r .ri i m Figure 2. Adjust Effects through Keyframes www.linuxjournal.com October 2007 | 63 FEATURE KDENLIVE KDENLIVE shows definite areas of improvement. It is the only NTSC-capable multitrack editor besides Cuisine (not cur¬ rently maintained) that is suitable for doing long-form projects in Linux (at least, without plopping down upwards of $ 1 5k for a used Discrete Smoke workstation). To date, I've used it on three long-form pro¬ jects, and the hours spent in front of it have been more than enough to shake out its better and worse points. On the credit side of the equation, it doesn't seem to suffer from the sound- sync problem introduced by unlocked DV audio (a format that is, mercifully, begin¬ ning to ebb away as HDV with its constant bit rate audio replaces it). In most open- source editors (and some commercial ones), the slightly variable sampling rate of DV audio was enough to cause gradual sync slippage, running into several seconds of slippage over the course of an hour- long DV tape. Good DV editors, like Kino, had internal resampling that kept things on track in the editor, but upon format conversion for a DVD, the sync problem would show up in spades. Kino fixed this problem by integrating audio locking upon export. Although KDENLIVE does not obviously do this, it nevertheless seems not to suffer from this problem—or, if it does, I haven't detected it yet. KDENLIVE's variety of export profiles also is worth cheering about. Presets (customizable) for most conceivable end formats are here, from Flash to H.264, from DV to DVD—a full range of standard- definition and Web profiles is here for the asking. The list of importable formats is no less impressive; because it uses FFMPEG for its encoding and decoding engine, it imports anything FFMPEG can read. This alone places KDENLIVE near the top of the heap for format compatibility. Finally, the list of available video effects and transitions, although limited, are well implemented and have sensible, accessible interfaces. The maintainers' determination to make a program that "just works" shines through in many areas, even at this early beta stage. A Work in Progress On the other side of the scale, KDENLIVE underperforms because of its unfinished status and some basic design oversights. The first set of issues will doubtless be fixed in future revisions, but the second are things not yet on the road map, or they are out-and-out software bugs. In the first category, the lack of high-definition profiles is a significant drawback; however, it is rumored that the next release of the software will fix this issue. The ability to create videos from still image files is non¬ functional at the moment due to a soft¬ ware bug, and the titler, while fully imple¬ mented, suffers from the same bug and thus doesn't actually create titles that show up in the viewer windows. The performance on the current SVN version, although respectable, is still sluggish enough to be irritating, and it could use some interface optimization. In terms of major oversights, three come readily to mind. The audio fader envelopes are not directly editable on the timeline, but may be accessed only through the effects control interface—a situation that is cumbersome at best. The faders are limited further by the ability to KDENLIVE’s variety of export profiles also is worth cheering about. insert only two keyframes in any given clip, so doing something as simple as a fade-out at the end of a clip necessitates slicing off a separate clip solely for the purposes of fading out—a definite work- flow impediment. Other effects suffer from similar limited keyframe-ability. Another oversight is the lack of dein¬ terlacing in the scaling work flow. This is significant when switching between resolu¬ tions and between aspect ratios in a single project, as if one does not deinterlace before down-scaling interlaced footage, significant unattractive artifacting results. At the moment, for example, the only way to edit interlaced HD footage is to convert it to a deinterlaced high bit rate H.264 before importing it into KDENLIVE. The same rigmarole is necessary for working with interlaced 16x9 SD footage. Although AVIDemux does this handsomely with HD footage, it's an incredible waste of time waiting for the computer to chew down all the raw footage rather than imple¬ menting deinterlacing into the output work flow as a check-box option. AVIDemux does not work on raw DV footage at all, so when working in standard-definition DV, one has to resort to hand-converting things on the command line or on (shudder) a Windows machine. Finally, nondestructive exporting does not seem to be implemented yet. In experiments I conducted with regular DV footage pushing through KDENLIVE, there was a noticeable quality drop between the footage I started with and the footage I wound up with at the other end of the pipeline. Dan Dennedy suggests to me that this may be a limitation with the underlying MLT framework, but the dig¬ ging I've done seems to indicate that the output is handled by FFMPEG scripts rather than by MLT. If this is indeed the case, tweaking the scripts should be all that's necessary to correct the problem. Other than those fairly obvious weak points, the only major drawback with KDENLIVE boils down to a failure of imagi¬ nation on the part of the interface design. The layout of the screens is entirely conventional and fails to leverage some of the more interesting innovations made by other open-source multitrack projects, such as Cuisine's multicamera track-switching paradigm and OpenMovieEditor's clip bin asset management strategy. KDENLIVE is building a solid base to work from, but as it grows, it should not neglect borrow¬ ing effective innovations from other open-source projects. In sum, KDENLIVE is a stable, usable multitrack editor that holds up well under the weight of long-form projects, so long as proper prep work is done to conform the footage before editing. Even though in beta stage, it's worth adding to one's toolbox and using often. Its limitations are significant, but forgivable at its current stage of develop¬ ment. Although not suitable for every pro¬ ject, it is ideal for long-format shows with basic editing needs. Hopefully, as it grows, it will expand its horizons and pilfer the good innovations of other nascent projects, and give us finally a powerful editing system suit¬ able for a wide range of projects. The devel¬ opers of this program deserve a pat on the back and a friendly beer, and hopefully per¬ ceive before them a wide vista for the future of their project.^ Dan Sawyer is the founder of ArtisticWhispers Productions (www.artisticwhispers.com), a small audio/video studio in the San Francisco Bay Area. He has been an enthusiastic advocate for free and open-source software since the late 1990s, when he founded the Blenderwars filmmaking community (www.blenderwars.com). Current projects include the independent SF feature Hunting Kestral and The Sophia Project, a fine-art photography book centering on strong women in myth. 64 | October 2007 www.linuxjournal.com INTEROP NEW YORK | OCTOBER 22-26, 2007 THE LEADING BUSINESS TECHNOLOGY EVENT • DATA CENTER • NETWORKING & • SECURITY SERVICES • ENTERPRISE 2.0 • UNIFIED • OPTIMIZATION COMMUNICATIONS • IP TELEPHONY • VOIP • VIRTUALIZATION • WIRELESS & MOBILITY • STORAGE • WI-FI WHERE THE TECHNOLOGY INDUSTRY MEETS TO DO BUSINESS See all of the latest technologies in action at Interop New York. Visit 200+ exhibitors, attend 100+ sessions and check out live demos of practical business solutions. Interop is the gathering place for business and IT leaders who want to find out what's next in business technology. Register today for your free Expo Pass.* Enter priority code MLSUNN41 at www.interop.com. *Free Expo Pass offer expires 10/19/07. Event is open to trade participants only. A business card demonstrating industry involvement will be required for entry. INDEPTH Interview with Bob Sutor IBM’s Bob Sutor opines about open standards, open source and how best to use software patents, glyn moody Bob Sutor studied mathematics at Harvard and Princeton before joining IBM Research in 1982. His exploration of ways of exchanging documents containing mathematical formulae so that they preserved their functionality led him to become active in the then-new field of XML. After a stint working on Web services and IBM's WebSphere family of application servers, he became Vice President, Open Source and Standards, in January 2005. GM: When did you first come across free software? BS: It was back in the mid-1990s. I was in IBM Research for 15 years, in the Math department there, and we had produced something called techexplorer, a plugin for Netscape Navigator that let you see [the results of the typesetting systems] TeX and LaTeX. We had come to the conclusion that we weren't going to get decent built-in math support in the browsers. In fact, this was the first Netscape plugin IBM ever produced, so it was kind of new and wild. As we wanted to move beyond a Windows implementation, we started looking at GCC. So we started trying to understand the intrica¬ cies of GPL, as it was back in the mid-1990s, and what exactly the rules were. [There were] some very, very early exchanges with Richard Stallman, just trying to figure out what this thing was and what was allowed. We maintained a fairly rapid e-mail exchange back and forth, until he felt I understood the situation. It was before IBM made its big push into Linux and made any sort of formal announcement, as we did a little bit later in the 1990s. I was a mathematician, who was doing a lot of software development. I certainly understood commercial software, but [free software] wasn't a deep philosophical stretch for me, coming from mathematics; I had never charged anybody for a lemma or a corollary. So it was very much trying to figure out how that fit into the IBM world, which at that point was completely commercial, traditional, proprietary software. 66 | October 2007 www.linuxjournal.com GM: When you became Vice President, Open Source and Standards, in January 2005, what did that involve? BS: Initially, a lot of it was around internal transformation. IBM is a very big place. We can tackle aspects of the same problem in many different parts of the company. And so when it came to standards, particularly things like industry-specific standards, we did a lot of work there to tie together the different parts of the company, to do the right thing in a very coordinated way. I have had, since I left research, a fairly vocal position. For example, when [the international standards body] OASIS really kicked in around 1999 or so, I was on the first board of directors there. I had been used to making a lot of noise, if you will, externally, about standards, about industry directions and so forth. And so in this current role, there are internal management aspects of it, as it relates to standards and open source across IBM, [and] there's also an external role as well. GM: What is IBM trying to achieve with open source? BS: From a business perspective, I would give a fairly traditional answer of saying, "Well, it's providing our customers with the appropriate IT tools and technologies to become better businesses." It may sound vanilla, but that really is the ultimate goal. But then you kind of back up and say, "Well, how do you do that?". You've got potentially a very large number of choices of software and ways you can mix and match them in together. You have traditional proprietary software and you have some open- source options as well. So for us, the business goal is to optimize this hybrid mix of proprietary and open-source software to give our customers what they need. GM: IBM was one of the first major computer companies to support open source, so why hasn't it followed Sun in pledging to open all of its code? BS: Well, let me say that Sun will do what Sun feels is appropriate for its business model and its history in the software business, and we will do the same. We will look at where we have successful businesses: if that's around proprietary software we will continue them, if that's what customers want to buy. Where customers want a more hybrid strategy, such as what we've done with WebSphere, we can do that too. If you go and you talk to our folks in the Linux Technology Center, they are just totally open source. We have the breadth that we don't feel that we have to just make some overarching, universal statement. We have a number of business models, some of which are traditional, some of which are completely open source—and increasingly, they're a combination of the two. So we do what works and, we hope, what our customers need us to do. Maybe something else works for Sun, but that, at least, is what's working for IBM. We're very willing to change these in different ways over time. GM: So if you thought you could serve your customers better and still make a profit, would you then be happy to make everything open source? BS: Well, I want to get away from this "everything open source", because I'm afraid people will extract just snippets of that discus¬ sion. We are doing more open source in 2007 than we did in ASA COMPUTERS Hie Exnert on Customized Seiners! www _ asacompute rs . a 1 -8QQ-REAL-PCS Hardware Systems for the open source community - Since 1989. (Linux, FreeBSD, NelBSD, GpenBSD, Solaris, MS etc.) T he AM D Opterd n proc esso rs deliver high-p erto rm a nee, scalable server solutions for the most advanced applications. "Runs both 32-amd $4-bit applications simultaneously". Your Custom Applicant* Solution!* "'Let us know your Needs../' "Wo will build you a Solution.../ AMD Opleron(TM) Value Server Starts d $B47 - 1U 14’ DeepfflW. -AMO Optercn 140 CPU, -512MB PC 32GQ DDR ECO Unbuffered - Support upto SCB DDR RAM - 40GB SATA Hard Disk - 2k 1QH0Q0 Mbps Lan. Quod AM D Optera n(TM) Server Jfarte at $2|12 - 1U AMD Optefon Model640. - 2GB Memory. Max 128 GB ■ Supports upto 64GB FBDIMM - 80 GB SATA II Hotswap Hard Drtve. ^Integrated Dual 10/1000 LAN Dud AMD OpteronfFM) Storage Stark at $4,120 - 5U Dual AMD Gpteron Model 246. - iSCSI or NAS Software Options. - Support upto 18TB of Storage. - Fail Hard Drive LED Indicator. Dual AMD Gpferon(TM) Storage Starh at $8,445 -8UAWID Opteran Model 246. - 4TB of Storage (36TB Max). - 1GB RAM - 2 x 10/100/1000 Gigabit LAN. - NAS or iSCSI Software Options. Why Do Business With ASA? "We Provide Approved EVAL Server,.," Since 1989. ASA has served customers like Cisco, Juniper, Caltech, FermilaP and most Universities We provide a total custom solution with OS of your choice Excellent pre and post-sales support. “Reliable hardware at the most competitive prices Please call or contact us for your next hardware purchase. 13 5 1 Ci lle D e l M undo. 5 i nti C l an. CA - 3 5 H54 wwv.iS3ionputers.ctm —E ma il : 5 t le5 gasaoompul e f s .com Tel: UOMEAL *- fcn ri wlaa SiSjrts rxjtvfrtd ica ktrcpiikta fcifiita rijn rms ?! tp pi; iiiTiis ri Iter isjriiw senra INDEPTH 2006, let's put it that way. And as we figure out how to serve our customers, we will go with that. IBM changes. What one considers IBM today, may change tomorrow. We do acquisitions, for example. We change our business models. We change our mix of what we offer customers, on a fairly con¬ stant basis. So I don't want to make one such strong statement. But let me say that if it makes good sense to get more involved with open source in certain areas, we have not been shy to do that in the past, and I don't anticipate our being shy to do that in the future. GM: Another important area of your work involves the OpenDocument Format (ODF). Where does ODF fit into IBM's overall strategy? BS: Well, there are two levels. On one hand, you can look at the very practical problem of interchange of information, as represented in office suite data formats. It's nice to represent them in XML; it's even better to represent that information in good, well-designed XML. What this means is that we get full fidelity of exchanging information between applications made by different people. They could be pro¬ prietary, they could be open source. The other aspect is that the fundamental model of how information is created and then shared is changing radically. This notion of a standalone office suite, at this point, is almost ancient in the software world: we've had office suites for well over a decade. To think that we will always use office suites the way we imagined them ten years ago, I think, is just silly, because we are seeing a shift to, for example, on-line applications, software as a service, exchange of information in ways that people never really imagined. That is, we don't always know who the users of our information are going to be, and we certainly don't expect them to be using exactly the same applications as we are. Indeed, they may not even be people, they may be new computer pro¬ grams that munch and do very fascinating things with this information. Therefore, breaking loose the data from any particu¬ lar application that happens to create it so it can be used by any other type of application, I think, is very important. Now, there's plenty of room for com¬ petition there. I don't expect people to completely stop using office suites, of course. But I do expect that people will simply compete on the quality and the price and the support of that category of software. That is, you don't get to just lock some people in to a particular prod¬ uct category because you happen to have a particular market share at a particular time. You continue to keep your cus¬ tomers by having high-quality products and support at the right price. If this leads to interchangeability—that is, the ability to use different applications on the same data—that's okay, because frankly, cus¬ tomers like that idea. A vendor or a software provider may not like this idea that you could possibly use somebody else's application, but from a customer perspective, that's pretty good. And, there are more customers than software providers, I may say. GM: We've seen some pretty interesting things happening around ODF since the commonwealth of Massachusetts stan¬ dardized on the format in September 2005. What do you think can be learned from that experience? BS: First, let me say that we have been pleas¬ antly surprised by almost everything that has to do with adoption of ODF. If you look his¬ torically, I can't name another standard that's gotten this much attention. [The] various attempts by people to say that open document standards are a good thing is all progress, because it represents awareness. We went from a situation where people never even thought of this issue to people actively thinking, "Well, yeah, maybe open document formats are a reasonable thing to have." Massachusetts brought a lot of attention to the issue. It really raised the profile of the importance of information, and who con¬ trolled the information. It comes down to a question of the sovereignty of the state of Massachusetts. What happened in 2005, as a result of this discussion around ODF, was this clear separation that said to vendors, "Thou shalt not tell governments what governments can and cannot do with this information." That was a historic turning point in the industry. It's funny, I had a couple of instances where I've spoken to rather senior people. They've said things like, "We'll have a discussion about ODF", or they'll say, "Well, everyone in my organization uses Microsoft Windows, and they have to use Microsoft Windows and Microsoft Office." And then right before the end of the conversation, almost as we're out the door, a person will slip in, "But of course, at home I use OpenOffice.org." What I think this is saying is that people at work in businesses and organizations are still bound by some of their traditional deci¬ sions. But a lot of the furor, the discussion around ODF, and [Microsoft's] OOXML, means people are actively looking at this again. And what I'm finding, in a more ad hoc way, is that when these very smart people in positions of power are making decisions about what to do with their own money on their own systems, you know, guess what they're choosing? And so if you look at any of the things like Massachusetts, or the study bill in Minnesota, or what came up in certain legis¬ latures this year and may come back next year, this is a matter of time. We are all in this for the very long haul. There is not going to be any one event that suddenly makes everyone on the planet use ODF, right? This is just relentlessly incremental, and it's all positive in different ways. GM: One of the key issues in the open- source world today is software patents. Shortly after you became Vice President, IBM made an unprecedented release of 500 patents to what you described as the patent commons. What was the background to this move? BS: Well, I got this job at the beginning of 2005. I think the first day of the week was a Tuesday because of the beginning of the year. Someone said, "You're briefing Forrester tomorrow on this patent pledge." And I said, "Oh really? What patent pledge is that?" And, in fact, it had been in the works for a little while. That was designed to shake up peo¬ ple's thinking about the use of software patents in open-source and proprietary software. It was a way of telling people, "Look, if we can make patents available to the Open Source community, and if maybe they can innovate and take them beyond what we have done with them, and then maybe we could use their results, that's a good thing. This is collaboration. And oh, by the way, no one has done this before; we're trying to figure this thing out too." That is, whatever you thought of the old IBM and what it's done with its patents, maybe you don't quite know what's 68 | October 2007 www.linuxjournal.com happening here. And, maybe we all need to rethink this. Maybe we are in some type of transition from our traditional view to some¬ thing new. We're not saying we have the answers. But we're saying that there are a number of things that one can do with intellectual property to accommodate both open-source and traditional software. GM: Many people, especially in the open-source world, believe that the patent system is seriously broken as far as software patents are concerned. What is IBM's position on that? BS: In part it's a philosophical issue. I haven't found any stone tablets anywhere that directly say yes or no in terms of patents. So I think it's a fairly social deci¬ sion as to whether people think they're good or bad. IBM has supported patents, where they are available, for the traditional reasons. But in ways I've described, I think we've been more flexible than almost anybody else in terms of its use. We wouldn't say that the system is broken. We would say that the system is under a lot of stress. And so it's for that reason we have been working with the Patent Office in the US and other places as well to just generally try to improve the system. There was in Europe, you may remem¬ ber, quite a big action around software and patents. And eventually, the bill was dropped because it had so many different things in it. But our position there at the time—and I think we're pretty consistent about this—is that software interoperability is extremely, extremely important. If you look at where we have gone royalty-free, for example—and royalty-free will typically translate to mean there can be open- source implementations—we have basically said that software patents should not get in the way of software interoperability, where it means data formats, where it means protocols and where it means APIs. GM: On a related note, I wondered what IBM's relationship with Novell was these days. BS: They're a strategic partner of ours—they were and are. GM: So what do you think of the agree¬ ment Novell and Microsoft announced in November 2006? BS: Speaking for IBM, we didn't feel it was necessary. We do not think anyone needs that particular protection from Microsoft. There's never been a successful suit against open source in any way, and so there was no particular need to strike such a deal. If it leads to the greater adop¬ tion of open source, the greater adoption of Linux, well, that's probably good for Novell. It's probably good from a partner situation. But we don't think that the world is fundamentally better off because that agreement was made. GM: Software patents are one of the key areas that GPLv3 attempts to address; what is IBM's view of the new license? BS: We think they did a great job. We think it was a tough task they set up for themselves. And Richard Stallman and the Free Software Foundation, they opened this up to the community—people who were fervently pro-free software, people in the middle with open source, as well as people who wanted to get involved who maybe didn't like the idea whatsoever. They completely opened themselves up for a tremendous amount of input from all sorts of groups. And through all of this, they had to look at the use cases. They had to under¬ stand philosophically what they were after and what they could practically get. And we think ultimately they played it pretty well. And so I certainly applaud what Richard's done, Eben Moglen as well, in getting to this [result]. I don't think anyone can deny that all the attention this got fundamentally points to the importance of free and open-source software, and the degree to which it is core to the IT industry today. If no one cared, no one would have said anything. It wouldn't have generated articles in the New York Times and the Wall Street Journal and BusinessWeek. And so, in many ways, free and open- source software has come of age. It's here to stay, and it's part of the mix.B Glyn Moody writes about open source at opendotdotdot.blogspot.com. WWW.SWELLSOFTWARE.COM 810-982-5955 nr/^ the leading GUI for Embedded Systems PEG+ - Full Featured Windowing in C++ C/PEG - Smallest Footprint in ANSI C Royalty Free Fast execution speed Completely ROM-able Delivered with Full Source Code Development Tools including FontCapture, PEG WindowBuilder, and ImageConvert Complete set of screen drivers included Completely customizable Industry leading RTOS Support Supports all popular target processors, video controllers and I/O devices Multi-lingual support - 2-byte character sets & UNICODE string encoding Event-driven programming model Application Design Services Knowledgeable and timely support to users around the globe Now includes a fully licensed version of Paint Shop Pro 9 www.linuxjournal.com October 2007 | 69 INDEPTH J The Ultimate Linux Home The LinuxMCE Project is a work in progress, promising finally to bring us the Home of the Future, jon "maddog" hall What started out as an article on the Ultimate Linux Multimedia System ended up being one on the Ultimate Linux Home—coming soon to a home near you, thanks to a project called LinuxMCE. What brought about this change of heart? When I was growing up, I was a big fan of maga¬ zines like Popular Mechanics and Popular Science that constantly talked about the Home of the Future. Such a home would manage lights, turning them on and off as you went from room to room. It also would control heating, automatically adjusting a room's temperature according to whether you were going to sleep, about to wake up or leaving for work. Figure 1. The Main Menu in the Plainest User Interface (All screenshots in this article courtesy of the LinuxMCE Project.) The Home of the Future appeared even more frequently in science- fiction books—the home would wrap its arms around you and take care of your every need. It would read books to you and let you answer the telephone without needing to hold a receiver. These services would be available throughout the house, tailored to the occupants' needs. During the years, such Home-of-the-Future qualities appeared again and again, mostly in concept homes. Small improvements came about through the use of interfaces, such as XI0 for remote control of lights and heat, but the promise of anything like the true Home of the Future still seemed far away, except for some with very expensive systems. About a year ago, I became aware of a project called Pluto (www.plutohome.com), which seemed to pull together elements of other projects, such as: ■ MythTV for recording TV programs, photos and music. ■ Xine for watching movies. ■ Asterisk for handling telephone calls. It also added security cameras and lighting control, and it tied them all together, controlling them from a variety of hardware, includ¬ ing Bluetooth-enabled phones. Not only that, but it also could "track" you as you walked from room to room and make sure the media play¬ ing in the room you were entering was the same as the media playing in the room you just left. You also could access the output of your security cameras from your cell phone and talk through the speakers of your sound system if you wanted to scare a burglar. The Pluto people made a very nice animated video showing how the system would work if it was completely implemented and installed correctly (plutohome.com/index.php?). They also gave a complete listing of all things it could do. It was awesome. Most important, Plutohome was FOSS! The people working on it encouraged others to join the project and take the code and put it on their own hardware or even to sell systems and services with Pluto software on them. They correctly pointed out that although there were people who would put their software onto "commodity" hard¬ ware, there were other people who wanted a complete and tested system, and they were happy to cater mostly to the second group while allowing the do-it-yourself people to use the software and help extend it. Although the functionality was interesting, I did not have the time to look at Pluto further, so I filed it away for future reference. Then, in March 2007, I heard about LinuxMCE and saw the video on Google Video (video.google.com/ videoplay?docid=-4422887272477313460&hl=en). At www.linuxmce.com, I read that Paul Webber, the founder of LinuxMCE, had been to CEDIA 2006, a consumer audio/video/home- automation tradeshow. There, he had seen a demonstration of Pluto, which by that time had been licensed to Monster. Paul was hooked, but when he was told that Pluto had all it could do to bring this technology to its own customers, Paul decided to work to make it more visible and more flexible for the Free Software community. The interesting part is that he did this with Pluto's blessing and support. Figure 2. The Same Main Menu, but Using the Most Powerful “Transparent” Interface 70 | October 2007 www.linuxjournal.com At the time of this writing, it is June 2007, and the project Paul thought "would take me a week" has been going on for five months, with Paul and another programmer "working on it almost full time", but they are approach¬ ing a usable version. They have done a lot of work to make it operate on "generic" PCs on top of a "generic" distribution and to make sure users would continue to receive all the device support they needed. Closed, single¬ purpose systems are not their target. Today, LinuxMCE is based on the Kubuntu distribution, with Aaron Seigo of KDE pledg¬ ing to integrate it even better. The goal of the LinuxMCE team is to have additional distribu¬ tions integrate and ship LinuxMCE. Imagine having a multimedia powerhouse as an option on every Linux desktop and notebook, and that every Linux desktop and notebook could optionally provide distributed, integrat¬ ed multimedia capabilities. What Distributed, Integrated Capabilities? LinuxMCE and Pluto are both frameworks that allow various components to be integrated together. Both projects utilize other major components along with a set of libraries called DCERouter (DCE stands for Data, Commands and Events), a general-purpose message router. It can send messages to applications that are "wrapped" in code and receive commands from those applications when they need something done. As an example of how this works, say a call comes into Asterisk while you are watch¬ ing TV. The Asterisk program could send a message to MythTV to mute the show or even halt the show until the call is finished. Of course, it is one thing when you think about programs controlling other programs, but if you apply almost the same logic to hardware devices, you more easily can map the functions of many home automation products into the framework of the system relatively effortlessly. The framework also manages plugins, like a lot of other flexible FOSS code (GIMP comes to mind), allowing more modules to be added for new functionality. Now, let's look at the different compo¬ nents that make up the system hardware. The Core The core is a server that can sit in an out-of- the-way place, such as your cellar or garage. The core is the place where the software and Figure 3. A File Browser in the Plainest User Interface data are stored. It should have two NIC cards: one for the internal network (recommended as at least 1GB) and one to go to the outside Internet (could be 10/100Mb). The core also should have an interface card for any existing telephone lines that you have (if you have any), for this also is the place where the Asterisk PBX is going to be managing your VoIP telephone calls. Alternatively, you could have a VoIP gateway at some other place in your network. The core should be a machine where you easily can add disk drives and probably have larger amounts of memory, depending on the load. Media Director The media director hooks up to your TV and audio/visual equipment. It can have inputs from your cable boxes or over-the-air connections, CD and DVD drives, audio input, digital "film" and other connections to amplifiers, TVs, recorders and so forth. There should be one media director for every room where you would consider having an "entertainment area". These media directors typically do not have any local storage. They boot PXE over the network. This allows you to use a PC for the media director and have an operating Total Linux Support Trustix Suse Ui STARTING AT 1GB DDR4QQ RAM - 160GB SATA2 HOD INTEL BOARDS & CPUS 100MBPS DEDICATED CISCO PORT -nnnCB Tumni inuiDi it iiun i inrn ■ UVWWU IMIIWV'II VI II IVLyULU CARI.NET/LJ ooo.ai i ,oyvi www.linuxjournal.com October 2007 | 71 INDEPTH 1 Figure 4. A File Browser in the Most Powerful “Transparent" Interface system stored on the local disk. Boot off the disk, and the operating system comes up. Boot over the network, and it becomes a media director. However, you also can install both the core and the media director software on your hard disk and start that software at any time. This hybrid allows you to boot into LinuxMCE, then stop the MCE software and use Kubuntu. You also could choose to boot Kubuntu, use it, and then start and stop the MCE software as you desire—very flexible. Orbiter The orbiter is a separate system that is just used to control the LinuxMCE system. It can have a touchscreen or use a keyboard and mouse. Orbiters can be a tablet system or even a regular PC, but the function is to control the various parts of the system. house or business. You also can speak to an intruder through your speaker systems—much better than a monitored alarm system. Finally, the core and the media directors actually are general-purpose computers and diskless clients. The core and media directors in your home are joined together as a network, but you also can combine multiple homes with an encrypted VPN. As you go house to house (or across the world), you can access your files and media. Imagine being in your hotel room with high-speed Internet and have access to all your home (or office) data. All of This Sounds Really Complex Well, let's just say that "Mom & Pop", my world-famous technophobic parents, probably will never set up a LinuxMCE system themselves. Even if they do manage to get the components out of the boxes, wired and set up, a lot of tailoring still has to be done for the whole system to run correctly. Floor plans have to be inserted, lighting fixtures have to be located, and a lot of data has to be imported or entered to make the whole thing work together. Fortunately, a very complete and systematic Web server also is built in to the system that lets you set and change almost any facility of the system remotely. In the case of Mom & Pop, once I visited their house, plugged in all of the systems and noted the MAC addresses of all the components, I actually could "tune" the system from my house in New Hampshire (and this is a blessing, believe me). Or, if I were a professional installer of this type of system, I could help customers by tuning their systems remotely. Now that you are all drooling and asking, "Where can I get it?", we continue with the rest of the story. Mobile Orbiter Symbian Bluetooth-enabled cell phones as well as some Linux, Windows and Windows CE Webpads (such as the Nokia 770), PDAs, Web browsers or even the CISCO 7970 phone can act as control interfaces for the system. As the documentation says, "You can use it [the CISCO 7970] to make calls and control your home as well." Symbian-based Bluetooth phones use Bluetooth for communication whenever they can and switch to the cellular data network when users are not close enough to use Bluetooth. If every media director has a Bluetooth connection, you probably will be within Bluetooth range in most parts of your house. The most amazing part is that the Bluetooth signal also allows the music or video that you select to follow you as you move from room to room. If two people with Bluetooth phones are in the same room, the choice of media stays with the first person who was in the room. If that person leaves, it switches to what the second person had chosen before entering the room. It also can set the lighting in that room, adjust the volume and other functions. The controls that show up on the various screens are tuned to indi¬ vidual people's tastes, "maddog" would have a different set of settings, music and preferences than "shedog" (if there were a "shedog"). And, when "maddog" goes to make a telephone call, it shows his own list of contacts. When controlling the system from a cell phone, it shows only the cell-phone owner's information. Home security also is managed by this system. You can integrate your video cameras into the system, and if someone breaks in to your house, a live feed sends an image to your cell phone and alerts you. You then can call 911 and make sure the authorities are headed toward your What Types of Hardware Do You Need? Coming up with the right hardware definitely is half the issue. Building one of these systems from cast-off hardware probably will be less than satisfying. The CPU probably will be used mainly for encoding and decoding video, particularly if you are interested in high-definition TV. If you have hardware encoders for this, a lot of the work is taken off the main CPU, but if you are using software encoders, you can expect to need about 1GHz of processing power (whatever that means these days) for every stream of video. Hardware encoders, such as the Hauppauge PVR series, unload quite a bit of CPU utilization and allow a single CPU to encode multiple streams at once. This would be useful if you wanted to record one program while Figure 5. Playing a DVD Remotely 72 | October 2007 www.linuxjournal.com Figure 6. Dialing a Phone Number watching another or record two programs at one time. On my system, it recognized my Hauppauge PVR-150 with no problems. If you are building the system yourself, be sure to study the LinuxMCE Project's documentation and the pages of MythTV's documentation that discuss hardware. Status of the LinuxMCE Project Pluto, as I stated previously, is aimed more at pre-packaged boxes with specialized hardware. The LinuxMCE Project is aimed more toward generalized hardware, with free and open-source software, and it uses no licensed, proprietary software at all. As such, a goal of LinuxMCE is to be as portable across all distribu¬ tions as possible, so all distributions can include it. Therefore, a signifi¬ cant amount of work had to be done by the development team, even if the Pluto people helped out a lot (which they did). The software is now going through testing to put it into VI .1 status, and even though it is only VI .1, the amount of functionality is staggering. In the beginning, it is prudent to understand that not every peripheral or video card will be supported and to try using only the hardware the testers in the forums have tested and found compatible. However, because the system is using major subsystems that have been around for a while, these projects do support a fairly well-known set of peripherals. Likewise, some of the installation and integration is not intuitive, even in the second beta test of version 1.1. On the other hand, if you purchase a box pre-installed, this won't be an issue, and at least one company is planning to offer pre-installed and supported systems (the support will be available by e-mail and phone). Bottom Line LinuxMCE is a large and complex project, but with a reasonable architecture to allow it to become a staple in the Free Software world. There are still a couple of rough edges, such as a missing editor to allow room diagrams to be inserted into the system easily, and (of course) the ever-necessary step-by-step documentation, but it has huge potential to help make free software more prevalent in the homes of average people. In my own LUG, we normally have 1 5 to 20 people show up at a meeting. When we recently had a MythTV meeting, 55 people showed up. LinuxMCE and similar projects move free software and Linux from computers to consumer appliances and make people more familiar with it. Although LinuxMCE still may not be "baked" enough for most people, I encourage the developers and the readers of this magazine to help move it forward. ■ Jon “maddog” Hall is the Executive Director of Linux International (www.li.org), a nonprofit association of end users who wish to support and promote the Linux operating system. During his career in commercial computing, which started in 1969, Mr Hall has been a programmer, systems designer, systems administrator, product manager, technical marketing manager and educator. He has worked for such companies as Western Electric Corporation, Aetna Life and Casualty, Bell Laboratories, Digital Equipment Corporation, VA Linux Systems and SGI. He is now an independent consultant in Free and Open Source Software (FOSS) Business and Technical issues. Data Acquisition & Control Computer iPac 9302 Cirrus Logic EP93Q2 ARM 200 Mhz Processor Floating Point Math Engine 2 USB 2.0 Host Ports SD/MMC Flash Disk Slot 0,1 48 Digital GRID Lines 1 10/100 Base-T Ethernet port 5 channels of 12 bit A/D & 3 PWMs 1 RS232 & 1 RS232/422/485 Serial Port Battery Backed Real Time clock/calendar Eclipse Linux Development Environment The iPac has enough I/O for demanding applications & with a size of 3.3” x 3.3” it can fit almost anywhere. Prices start at $150,00. Please contact us for more information. [ SiiiLt 19851 lilAC 22" - mC Sa Equipment Monitor And Control Phone: (618) 529-4525 * Fax: (816) 457-0110 * Web: www.emacinc.com www.linuxjournal.com October 2007 | 73 INDEPTH J Stream Control Transmission Protocol (SCTP) Associations SCTP network protocol JAN NEWMARCH This second in a series of articles on the examines associations and connections. An SCTP association is a generalisation of a TCP connection. Usually a TCP connection is one-to-one between two network inter¬ faces, one on a server and the other on a client. In contrast, an SCTP association is many-to-many in two ways: ■ Multiple network interfaces on a server can be associated with multiple interfaces on a client. For example, suppose the server and client both have an Ethernet card and a Wi-Fi card connected to the Internet. Then, data can flow over a single association in up to four possible ways: Ethernet to Ethernet, Ethernet to Wi-Fi, Wi-Fi to Ethernet or Wi-Fi to Wi-Fi. ■ An association also can carry multiple logical streams. These are numbered from zero upward. So, for example, stream zero could carry control instructions, while stream one could carry small pieces of data (such as small files), and stream two could carry larger pieces of data (such as an MPEG movie). The three streams are logically independent, so that delays on one stream do not cause delays on any other stream. Note that a single socket can have multiple associations—that is, one socket can be used to talk to multiple other hosts. In general, these different associations are distinguished by having an association ID. The socket API for SCTP distinguishes between situations where exactly one association can exist (a one-to-one socket) or where a socket can manage many associations (a one-to-many socket). The first case corresponds to the TCP-like case that I discussed in the first article on SCTP [ LJ , September 2007], The second case will be covered in the next article. In this article, I look only at a single association, which is applicable to both the one-to-one and the one-to-many sockets. Using a Subset of Network Interfaces TCP and UDP use a single network interface on an endpoint, by speci¬ fying its IP address in a socketaddr structure. If you specify the wild¬ card address INADDR_ANY, a server will listen on all interfaces while a client will choose only one. In any case, communication takes place only between a single interface on each endpoint. As an aside, if you want to know what all the interfaces on your machine are, use the call ioctl() with parameter SIOCGIFCONF. How to do this is described in WR Stevens et al., Unix Network Programming , vol 1, section 17.5. Using only one interface reduces reliability when more than one is available. A network cable may have poor connections, or you may be too far from a wireless access point for a reliable signal. On the other hand, using all of the interfaces may not always be desirable. For example, in Australia, the download charges for 3-G or WiMAX connections are ridiculously expensive, so you would use that interface only if no others were available. Or, a bridge would expose the internal and external interfaces separately to different groups of users. SCTP allows an application to choose a subset of interfaces on either the source or destination side of an association. Some implementations also will allow interfaces to be added or removed dynamically, so the application can adjust to different states of the network connections. By registering for association-change events (which will be discussed in the next article), one endpoint can track changes in the interfaces at the other end. The normal socket call bind() just takes a single sockaddr parameter to bind the socket to a single IP address (or to the wildcard address). SCTP extends this by introducing a new call, sctp_bindx(), which takes an array of sockaddrs to bind the socket to all of these addresses. The socket is bound only to a single port though; all of the port numbers in the array of sockaddrs must have the same port number. And, if addresses are added or removed later, they must have the same bound port value. Otherwise, the call fails. There is another wrinkle to sctp_bindx() regarding IPv4 and IPv6 addresses. The socket can be passed a set of only IPv4 sockaddrs, a set of only IPv6 sockaddrs or a mixture of both. The two types of socket address structures, sockaddrjn and sockaddr_in6, have different sizes, so mixing these in the same array can cause alignment issues. SCTP packs the structures together with no wasted space between them. So, you can't just use an index into the array, you have to copy the right number of bytes for each structure and then move up by that amount. The call to bind a set of addresses is: int sctp_bindx(int sd, struct sockaddr *addrs, ^int addrcount, int flags) where flags can be one of SCTP_BINDX_ADD_ADDR or SCTP_BINDX_REM_ADDR, and the second parameter is a packed list of IPv4 and IPv6 socket address structures. It is relatively easy to use this on a server to allow clients to con¬ nect on any of the server's bound interfaces. But, how do you do this on a client? Where is the bind() operation? Well, just like TCP, this is hidden under the covers. In TCP, if a call to connect() is made and the socket is not yet bound (the usual case for a client), the TCP stack will choose one of the interfaces and an ephemeral port. You can make an 74 | October 2007 www.linuxjournal.com Listing 1. multi_homed_client.c #include #include #include #include #include #include addr.sin_port = 0; if (bind(sockfd, (struct sockaddr *) &addr, addr_size) = -1) { perror("bind"); exit(l); } /* this gets sin.sin_port so we can find the ephemeral port */ getsockname(sockfd, (struct sockaddr *) &addr, &addr_size); port = addr.sin_port; printf("Ephemeral port is %d\n", port); #include int main(int argc, char *argv[]) { int sockfd; int n; struct sockaddr_in addr, *addresses; int addr_size = sizeof(struct sockaddr_in); int addr_count = argc - 2; int port; if (argc < 2) { fprintf(stderr, "Usage %s client-addresses...\n", argv[0]); exit(l); } /* create endpoint */ sockfd = socket(AF_INET, SOCK_STREAM, IPPR0T0_5CTP); if (sockfd < 0) { perror("socket"); exit(2); } addresses = malloc(addr_size * addr_count); if (addresses == NULL) { perror("malloc"); exit(l); } /* do bind to get ephemeral port first */ addr.sin_family = AF_INET; addr.sin_addr.s_addr = inet_addr(argv[l]); explicit call to bind() that will let you choose the interface, but usually the port is left as zero, so an ephemeral port is still chosen. You can do exactly the same with SCTP—don't call bind() and leave it to the SCTP stack. This will choose an ephemeral port like TCP, but instead of using a single interface, it will choose a set of interfaces (probably all that are available). So calling connectO without an initial bind() or sctp_bindx() will give you multihoming on the client side automatically. If you call bind() with a specified interface before connectO in the client, you get only that single client-side interface, losing one of the advantages of SCTP! If you call bind() with the wildcard address INADDR_ANY, SCTP will choose a set of interfaces for you. So, SCTP will try to give you multihoming unless you pin it down for ( n = 2; n < argc; n++) { addr.sin_family = AF_INET; addr.sin_addr.s_addr = inet_addr(argv[n]); addr.sin_port = port; memcpy(addresses + (n-2), &addr, addr_size); } if (sctp_bindx(sockfd, (struct sockaddr *) addresses, addr_count, SCTP_BINDX_ADD_ADDR) == -1) { perror("sctp bindx"); exit(2); } /* get local list */ addr_count = sctp_getladdrs(sockfd, 0, (struct sockaddr**)&addresses); for (n = 0; n < addr_count; n++) { memcpy(&addr, addresses+n, addr_size); printf("addr %s, port %d\n", inet_ntoa(addr.sin_addr.s_addr), addr.sin_port); } /* we don’t actually connect to any server in this program */ close(sockfd); exit(0); } to a single address using bind() or to a specific set of addresses using sctp_bindx(). With SCTP, I would expect a call to sctp_bindx() with all ports set to zero to choose the same ephemeral port for all addresses. Instead, the current Linux implementation (up to kernel 2.6.21) gets an ephemeral port for the first address and then throws an error, because the ports in the later addresses are still zero instead of this ephemeral value. The workaround is to call bind() with one address with port zero, see what the system set the port to, and then call bindx() on the remaining addresses with this new port number. Listing 1 (multi_homed_client.c) shows an example of this. This workaround probably will become unnecessary in the next specification of SCTP following discussion on the SCTP mailing list. www.linuxjournal.com October 2007 | 75 INDEPTH 1 Listing 2. streamcount_echo_client.c #include #include #include #include #include #include #include #include #define ECHO_PORT 2013 char *usage_msg = "usage: astreamcount_echo_client ip-addr istreams ostreams"; char *msg = "hello"; void usage() { fprintf(stderr, "%s\n", usage_msg); exit(1); int main(int argc, char *argv[]) { int sockfd; int len; struct sockaddr_in serv_addr; int port = ECH0_P0RT; struct sctp_initmsg initmsg; struct sctp_status status; if (argc != 4) usage(); /* create endpoint */ sockfd = socket(AF_INET, SOCK_STREAM, IPPR0T0_5CTP ); if (sockfd < 0) { perror("socket creation"); exit(2); } /* connect to server */ serv_addr.sin_family = AF_INET; serv_addr.sin_addr.s_addr = inet_addr(argv[1]); serv_addr.sin_port = htons(port); memset(&initmsg, 0, sizeof(initmsg)); initmsg.sinit_max_instreams = atoi(argv[2]); initmsg.sinit_num_ostreams = atoi(argv[3]); printf("Asking for: input streams: %d, output streams: %d\n", initmsg.sinit_max_instreams, initmsg.sinit_num_ostreams); if (setsockopt(sockfd, IPPR0T0_SCTP, SCTP_INITMSG, &initmsg, sizeof(initmsg))) { perror("set sock opt\n"); } if (connect(sockfd, (struct sockaddr *) &serv_addr, sizeof(serv_addr)) < 0) { perror("connectx"); exit(3); } len = sizeof(status); memset(&status, 0, len); if (getsockopt(sockfd, IPPR0T0_SCTP, SCTP_STATUS, &status, &len) == -1) { perror("get sock opt"); } printf("Got: input streams: %d, output streams: %d\n", status.sstat_instrms, status.sstat_outstrms); /* give the server time to do something */ sleep(2); /* no reads/writes are done */ close(sockfd); exit(0); Using Multiple Interfaces You can set the local interfaces to be used by sctp_bindx(). A client also can specify the subset of addresses that it wants to use to connect to the server by using the call sctp_connectx(), which takes a list of socket address structures just like sctp_bindx(). Why do this? Well, using connect() with a single address is a possible point of failure at the time that the initial connection is done. This is what the function sctp_connectx() solves. It allows the client to try multiple addresses in order to connect to the server. The set of addresses in sctp_connectx() is used just to make the initial connection. But, after the connection is established, an interchange of information takes place between the two end¬ points. In that exchange, the remote peer tells the local peer which addresses it actually wants to use and vice versa. The set of remote addresses that the remote peer will use need not be the same as what the client used in the connection. However, you at least can assume that one (but you don't know which one) of the addresses passed to sctp_connectx() will appear in the list 76 | October 2007 www.linuxjournal.com that the remote peer offers, because the local client had to connect to something! So, if the remote peer chooses the set of addresses it uses, how does the local client find which ones they are? This is done by another function, sctp_getpaddrs(), that gives the set of remote peer addresses. There is also an sctp_getladdrs() function, in case the local peer forgets which addresses it is using! Once an association is set up between two endpoints, messages can be sent between them. Note that SCTP does not concern itself with QoS (Quality-of-Service) issues, such as real-time delivery, but only with reliability issues. SCTP uses the multihomed capabilities to try as many possible routes as possible to get messages through. So on the sending side, there is no control over which interfaces are used; indeed, the sender might even use a scheme such as round- robin among its interfaces for each message. However, the sending application can indicate to its SCTP stack which of the remote peer's interface it would prefer to use, and it can tell the remote peer on which interfaces it would prefer to receive messages. These are done by using the setsockoptO call with option type as SCTP_PRIMARY_ADDR or SCTP_SET_PEER_PRIMARY_ADDR. Of course, if these particular addresses are not available, SCTP simply will use different addresses in the association. Once SCTP is told which interfaces to use, it basically looks after things itself. It uses heartbeats to keep track of which interfaces are alive, and it switches interfaces transparently when failure occurs. This is to satisfy the design goals of SCTP for improved reliability over TCP. Applications can give hints to the SCTP stack about which interfaces to use, but the stack will ignore these hints on failure. Streams In TCP, a stream is just a sequence of bytes. In SCTP, it has a different meaning; a stream is a logical channel along which messages are sent, and a single association can have many streams. The original motivation for streams came from the telephony domain, where multiple reliable channels were needed, but the messages on each channel were independent of those on other channels. In last month's article, we pointed out some TCP applications that could benefit from streams, such as FTP, which uses two sockets for data and control messages. In addition, an increasing number of applications are multithreaded, and streams open up the possibility of a thread in one peer being able to communicate with a thread in another peer without worrying about being blocked by messages sent by other threads. The socket I/O calls read/write/send/recv do not know about SCTP streams. By default, the write calls all use stream number zero (this can be changed by a socket option), but the read calls will read messages on all streams, and there is no indication as to which stream is used. So, to use streams effectively, you need to use some of the I/O calls that are designed specifically for SCTP. Negotiating the Number of Streams Each endpoint of an association will support a certain number of streams. A Linux endpoint, by default, will expect to be able to send to ten streams, while it can receive on 65,535 streams. Other SCTP stacks may have different default values. These values can be changed by setting the socket option SCTPJNITMSG, which takes a structure sctpjnitmsg: Linux - FreeBSD - x86 Solaris - MS etc. Proven technology. Proven reliability. When you can’t afford to take chances with your business data or productivity, rely on a GS-1245 Server powered by the Intel® Xeon® Processors. 2 Nodes & Up to 16 Cores - in 1U Ideal for high density clustering in standard ill form factor. Upto 16 Cores for high CPU needs. Easy to configure failover nodes. Features: - 1U rack-optimized chassis (1 r 75in.) - Up to 2 Quad Core Intel® Xeon® Wood crest per Mode with 1333 MHz system bus - Up to 16 Woodcrast Cores Per 1U racks pace - Up to 32GB DPR2 66? & 533 SDRAM Fully Suffered DIMM (FB-DIMM) Per Mode - Dual-port Gigabit Ethernet Per Node - 2 SATA Removable HDD Per Node - 1 (x8) PCI-Express Per Node Servers :: Storage :: Appliances Genstor Systems, Inc. 780 Montague Express. # 604 i San Jose, CA 06131 - Www.genstorcom Er ; m a il: sa l es@ gen s tbr.com Hi Phone: 1-877-25 SERVER or 1^08-383-0120 Imete hub**: Xeenfr. inatd*® 1 an trwlenwris or of Inurf CoqKjoiUqri « ns iubaidiLMwa in am Uniind Sums* And lad'ior oountaun. 1 struct sctp_initmsg { uintl6_t sinit_num_ostreams ; point during association initialisation. Each endpoint in an association will have an idea of how many uintl6_t sinit_max_ostreams; input and output streams it will allow on an association, as dis¬ uintl6_t sinit_max_attempts; uintl6_t sinit_max_init_timeo; cussed in the previous paragraph. During the establishment of the association, the endpoints exchange these values. Negotiation of } final values is just a matter of taking the minimum values. If one end wants 20 output streams, and the other wants only 10 input If this socket option is used to set values, it must be done before an association is made. The parameters will be sent to the peer end- streams, the result is the smaller, 10, and similarly for the number of streams in the opposite direction. Listing 3. streamcount_echo_server.c #include perror("sctp bind"); #include exit(2); #include } #include #include #include memset(&initmsg, 0, sizeof(initmsg)); initmsg.sinit_max_instreams = atoi(argv[1]); #include initmsg.sinit_num_ostreams = atoi(argv[2]); printf("Asking for: input streams: %d, output streams: %d\n", #define ECH0_P0RT 2013 initmsg.sinit_max_instreams, initmsg.sinit_num_ostreams); char *usage_msg = "usage: streamcount_echo_server istreams ostreams"; if (setsockopt(sockfd, IPPR0T0_SCTP, void usage() { SCTP_INITMSG, &initmsg, sizeof(initmsg))) { fprintf(stderr, "%s\n", usagejnsg); perror("set sock opt\n"); exit(l); \ } / int main(int argc, char *argv[]) { /* specify queue */ listen(sockfd, 5): for (;;) { int sockfd, client_sockfd; len = sizeof(client_addr); int len; client_sockfd = accept(sockfd, (struct sockaddr *) struct sockaddr_in serv_addr, client_addr; &client_addr, &len); int port = ECH0_P0RT; if (client_sockfd == -1) { struct sctp_initmsg initmsg; perror(NULL); continue: struct sctp_status status; } if (argc ! = 3) usageQ ; memset(&status, 0, sizeof(status)); len = sizeof(status); /* create endpoint */ if (getsockopt(client_sockfd, IPPR0T0_SCTP, sockfd = socket(AF_INET, SOCK_STREAM, SCTP_STATUS, &status, &len) == -1) { IPPR0T0_SCTP perror("get sock opt"); i ) . if (sockfd < 0) { / printf("Got: input streams: %d, output streams: %d\n", perror("socket"); status.sstat_instrms, exit(2); \ status.sstat_outstrms); i serv_addr.sin_family = AF_INET; /* give the client time to do something */ sleep(2); serv_addr.sin_addr.s_addr = INADDR_ANY; close(client_sockfd); serv_addr.sin_port = htons(port); if (bind(sockfd, (struct sockaddr *) &serv_addr, / } sizeof(serv_addr)) == -1) { 78 | October 2007 www.linuxjournal.com Listing 4. streamsend_echo_client.c #include #include #include #include #include #include #include #include #define SIZE 1024 char buf[SIZE]; #define ECHO_PORT 2013 char *usage_msg = "usage: streamsend_echo_client ip-addr istreams ostreams stream"; void usage() { fprintf(stderr, "%s\n", usage_msg); exit(l); } int main(int argc, char *argv[]) { int sockfd; int len; struct sockaddr_in serv_addr; struct sockaddr_in *addresses; int addr_size = sizeof(struct sockaddr_in); int addr_count = argc - 1; int port = ECH0_P0RT; char *message = "hello\n"; struct sctp_initmsg initmsg; struct sctp_status status; struct sctp_sndrcvinfo sinfo; int ochannel; if (argc != 5) usageQ; /* create endpoint */ sockfd = socket(AF_INET, SOCK_STREAM, IPPR0T0_SCTP ); if (sockfd < 0) { perror(NULL); exit(2); } /* connect to server */ addresses = malloc(addr_size); if (addresses == NULL) { exit(l); } serv_addr.sin_family = AF_INET; serv_addr.sin_addr.s_addr = inet_addr(argv[l]); serv_addr.sin_port = htons(port); memcpy(addresses, &serv_addr, addr_size); memset(&initmsg, 0, sizeof(initmsg)); initmsg.sinit_max_instreams = atoi(argv[2]); initmsg.sinit_num_ostreams = atoi(argv[3]); printf("Asking for: input streams: %d, output streams: %d\n", initmsg.sinit_max_instreams, initmsg.sinit_num_ostreams); if (setsockopt(sockfd, IPPR0T0_SCTP, SCTP_INITMSG, &initmsg, sizeof(initmsg))) { perror("set sock opt\n"); } if (sctp_connectx(sockfd, (struct sockaddr *) addresses, 1) < 0) perror("connectx"); exit(3); } memset(&status, 0, sizeof(status)); len = sizeof(status); status.sstat_assoc_id = 1; if (getsockopt(sockfd, IPPR0T0_SCTP, SCTP_STATUS, &status, &len) == -1) { perror("get sock opt\n"); } printf("Got: input streams: %d, output streams: %d\n", status.sstat_instrms, status.sstat_outstrms); /* sanity check channel */ ochannel = atoi(argv[4]); if (ochannel >= status.sstat_outstrms) printf("Writing on illegal channel %d\n", ochannel); /* transfer data */ bzero(&sinfo, sizeof(sinfo)); sinfo.sinfo_stream = ochannel; sctp_send(sockfd, message, strlen(message), &sinfo, 0); sinfo.sinfo_flags = SCTP_E0F; sctp_send(sockfd, NULL, 0, &sinfo, 0); close(sockfd); exit(0); www.linuxjournal.com October 2007 | 79 INDEPTH 1 Listing 5. streamsend_echo_server.c #include #include #include #include #include #include #include #define SIZE 1024 char buf[SIZE]; #define TIME_PORT 2013 char *usage_msg = "usage: app ip-addr istreams ostreams"; void usage() { fprintf(stderr, "%s\n", usage_msg); exit(l); } int main(int argc, char *argv[]) { int sockfd, client_sockfd; int nread, len; struct sockaddr_in serv_addr, client_addr; time_t t; struct sockaddr_in *addresses; int addr_size = sizeof(struct sockaddr_in); int addr_count = 1; int port = TIME_P0RT; int n; struct sctp_initmsg initmsg; struct sctp_status status; sctp_assoc_t associd; struct sctp_sndrcvinfo sinfo; struct sctp_event_subscribe events; int flags; if (argc != 4) usageQ; /* create endpoint */ sockfd = socket(AF_INET, SOCK_STREAM, IPPR0T0_SCTP ); if (sockfd < 0) { perror(NULL); exit(2); } serv_addr.sin_family = AF_INET; serv_addr.sin_addr.s_addr = inet_addr(argv[l]); serv_addr.sin_port = htons(port); if (sctp_bindx(sockfd, (struct sockaddr *) &serv_addr, addr_count SCTP_BINDX_ADD_ADDR) == -1) { perror("sctp bindx"); exit(2); } memset(&initmsg, 0, sizeof(initmsg)); initmsg.sinit_max_instreams = atoi(argv[2]); initmsg.sinit_num_ostreams = atoi(argv[3]); printf("Asking for: input streams: %d, output streams: %d\n", initmsg.sinit_max_instreams, initmsg.sinit_num_ostreams); if (setsockopt(sockfd, IPPR0T0_5CTP, SCTP_INITMSG, &initmsg, sizeof(initmsg))) { perror("set sock opt\n"); } /* specify queue */ listen(sockfd, 5); for (;;) { len = sizeof(client_addr); client_sockfd = accept(sockfd, (struct sockaddr *) &client_addr &len); An endpoint will need to know how many output streams are available for writing in order not to exceed the limits. This value is determined during association setup. After setup, the endpoint can find this by making a query using getsockopt(). However, there is a little wrinkle here: a socket may have many associations (to different endpoints), and each association may have set different values. So, we have to make a query that asks for the parameters for a particular association, not just for the socket. The parameter to ask for is SCTP_STATUS, which takes a structure of type sctp_status: sctp_assoc_t sstat_assoc_id; int32_t sstat_state; uin 13 2_t sstat_rwnd; uintl6_t sstat_unackdata; uintl6_t sstat_penddata; uintl6_t sstat_instrms; uintl6_t sstat_outstrms; uint32_t sstat_fragmentation_point; struct sctp_paddrinfo sstat_primary; struct sctp_status { This has fields sstatjnstrms and sstat_outstrms, which contain 80 | October 2007 www.linuxjournal.com Advertiser Index if (client_sockfd == -1) { perror(NULL); continue; } memset(&status, 0, sizeof(status)); len = sizeof(status); status.sstat_assoc_id = 0; if (getsockopt(client_sockfd, IPPR0T0_SCTP, SCTP_STATUS, &status, &len) == -1) { perror("get sock opt\n"); } printf("Got: input streams: %d, output streams: %d\n", status.sstat_instrms, status.sstat_outstrms); for(;;) { /* transfer data */ len = sizeof(client_addr); bzero(&sinfo, sizeof(sinfo)); nread = sctp_recvmsg(client_sockfd, buf, SIZE, (struct sockaddr *) &client_addr, &len, &sinfo, &flags); if (nread == 0) { break; } printf("read %d bytes on channel %hd\n", nread, sinfo.sinfo_stream); printf("sinfo flags: %d\n", sinfo.sinfo_flags); write(l, buf, nread); } close(client_sockfd); } } the required information. See Listings 2 and 3 for a client and server negotiating the number of streams in each direction. Association ID For the one-to-one socket we discussed in the first article in this series, there can be only one association at any time. For the one-to-many sockets we will cover in the next article, there can be many associations active at any one time—a peer can be connected to many other peers simultaneously. This is different from TCP where only one connection on a socket can exist and also is different from UDP where no connections exist and messages are just sent to arbitrary peers. For advertising information, please contact our sales department at 1-713-344-1956 ext. 2 or ads@linu)gournaLcom. www.linuxjournal.com/advertising Advertiser Page# Advertiser Page# Aberdeen, LLC www.aberdeeninc.com EH LinuxWorld UK www.linuxworldexpo.co.uk ED APPRO HPC Solutions appro.com m Logic Supply Inc www.logicsupply.com ED ASA Computers www.asacomputers.com |67|85| LPI www.lpi.org ED Avocent Corporation www.avocent.com/remotecontrol □ Microway, Inc. www.microway.com ED Cari.net www.cari.net ED Polywell Computers, Inc. www.polywell.com m Coraid, Inc. www.coraid.com □ The Portland Group www.pgroup.com ED Coyote Point www.coyotepoint.com □ Qsol.Com www.gsol.com □ CSI 2007 - Confex Partners Ltd. www.csinetsec.com EH RISoft, Inc. www.r1soft.com B Emac, Inc. www.emacinc.com ED Rackspace Managed Hosting www.rackspace.com ED EmperorLinux www.emperorlinux.com EH R Cubed Technologies www.rcubedtech.com E3 FairCom www.faircom.com m SC 2007/Hall-Erickson, Inc. sc07.supercomputing.org OH Gecad Technologies/Axigen www.axigen.com □ Servers Direct www.serversdirect.com □ Genstor Systems, Inc. www.genstor.com m Silicon Mechanics www.siliconmechanics.com ll 3, |l 71 HPC Systems, Inc. www.hpcsystems.com □ Super Micro Computer, Inc. www.supermicro.com ED Hurricane Electric www.he.net 0 Swell Software, Inc. www.swellsoftware.com EEI Intel www.intel.com ED Technologic Systems www.embeddedx86.com 0 Interop New York 2007 www.interop.com ED Uniwide Technologies www.uniwide.com ED Iron Systems www.ironsystems.com ED USENIX Annual Technical Conference www.usenix.org/lisa07/lj ED ISPCON www.ispcon.com B Verio www.verio.com ED King Star Computer www.kingstarusa.com ED White Oak Technologies www.woti.com B Linux Journal www.linuxjournal.com 0 www.linuxjournal.com October 2007 | 81 INDEPTH 1 When there can be many associations, you need to be able to distinguish between them. This is done by an opaque data type called an association ID. You need to use this sometimes, but not every time. For one-to-one sockets, there is only one association, so the association ID is always ignored. For one-to-many sockets, when the association is "obvious", the association ID again is ignored. This occurs, for example, when you write to a peer and give the peer's socket address; there can be only one association to a peer (but many associations to many peers), so if the peer is known, the association is known, and there is no need for the ID. But the association ID has to be used when the SCTP stack cannot work out for itself which association is meant. One place where this happens is in the getsockoptO call described previously to find the number of streams of an association on a one-to-many socket. I will defer the discussion of how to find the association ID to the next article, where I look at one-to-many sockets. Writing to and Reading from a Stream There are several ways of writing to a stream and telling to which stream a read belongs. Some of them make use of a structure of type sctp_sndrcvinfo: struct sctp_sndrcvinfo { uintl6_t sinfo_stream; uintl6_t sinfo_ssn; uintl6_t sinfo_flags; uint32_t sinfo_ppid; uint32_t sinfo_context; uint32_t sinfo_timetolive; uint32_t sinfo_tsn; uint32_t sinfo_cumtsn; sctp_assoc_t sinfo_assoc_id; } Most of the fields in this structure are not of interest to us at the moment. The interesting one is the first one, sinfo_stream. To write to a particular stream, zero out all fields and set this one; to read, zero out all fields again, do the read, and then examine this field. (As an aside, if the SCTP stack cannot work out which association is meant, the last field, sinfo_assoc_id, must be set.) The function call to write a message is: int sctp_send(int sd, const void *msg, size_t ten, const struct sctp_sndrcvinfo *sinfo, int flags); where the field sinfo_stream of sinfo has been set. The call to read is, conversely: ssize_t sctp_recvmsg(int sd, void *msg, size_t ten, struct sockaddr *from, socklen_t *fromlen struct sctp_sndrcvinfo *sinfo int *msg_flags) The stream number is then available in sinfo.sinfo_stream. The SCTP stack keeps a lot of information about each message that passes between peers. It also keeps information about the state of each association. To avoid overloading applications, most of this information is suppressed and is not passed to the application. In particular, by default, the structure sctp_sndrcvinfo is not filled in, so a reader cannot tell on which stream a read occurred! To enable this to be filled, a socket option must be called first as: struct sctp_event_subscribe events: bzero(&events, sizeof(events)); events.sctp_data_io_event - 1; setsockopt(sockfd, IPPR0T0_SCTP, SCTP_EVENTS, &events, sizeof(events)); (More details on SCTP events will be given in the next article.) See Listings 4 (streamsend_echo_client.c) and 5 (streamsend_echo_server.c) for an example of a client and server using a specific stream for communication. There is no way to specify from which stream to read. This is deliberate; the intention is that when data is ready on any stream, then you read it. Otherwise, data could be blocked on a stream with no one to read it, which eventually could fill up system buffers. So, you can't restrict reading to any particular stream. But, once a read is done, you can tell which stream it has come from by using the mechanism above. Typically, a server that reads and handles a message will have (pseudocode) that looks like this: while (true) { nread = sctp_recvmsg(..., msg.&sinfo, ...) if (nread <= 0) break; assoc_id = sinfo.sinfo_assoc_id; stream = sinfo.sinfo_stream; handle_mesg(assoc_id, stream, msg, nread): } This is a single-threaded read loop. It ensures that information is read, no matter what association or stream it is sent on. The applica¬ tion function handle_mesg() can, of course, dispatch the message to different threads if it wants. Writes, on the other hand can be sent from multiple threads if desired. Conclusion This article has discussed a feature novel to SCTP, streams. A stream allows multiple data channels on a single association. This avoids a major problem of TCP, head-of-line blocking, but it also allows applica¬ tions that deal with multiple logical streams to be written more easily. The next and final article will look at how SCTP can handle multiple associations at once, simplifying the TCP model and also offering improvements over the UDP connection model. ■ Jan Newmarch is Honorary Senior Research Fellow at Monash University. He has been using Linux since kernel 0.98. He has written four books and many papers and given courses on many technical topics, concentrating on network programming for the last six years. His Web site is jan.newmarch.name. 82 | October 2007 www.linuxjournal.com Reno-Sp Conve Reno, N November SC07, , the premier international conference on high performance computing, networking, storage, and analysis, will convene in November 2007 in Reno, Nevada. SC07 expands the frontiers of those high performance technologies with a new initiative in high perform¬ ance clusters. Of course, SC07 will also host the exceptional technical program, tutorials, workshops, exhibits, education program, and other exciting activities you've come to expect with the SC conferences. SC is the one place attendees can see tomorrow's technology being used to solve world- class challenge problems today. This series has become a unique gathering spot for international scientists, researchers, software developers, network engineers, policy makers, corporate managers, CIOs, and IT administrators. SC07 represents the latest and greatest in technology, applications, industrial products, research results, and national/international initiatives. We look forward to your participation! Conference Location: Reno, Nevada Conference Dates: November 10-16, 2007 Exhibition Dates: November 13-15, 2007 For more information, visit: http:/ / sc07.supercomputing.org Sponsors: ACM SIGARCH/IEEE Computer Society 4ieee Computer society INDEPTH J Multimedia Dynamite An overview of the awesome power and configurability of MPlayer. girish venkatachalam What should the Linux movie player of your dreams do? It should play any movie/video that you throw at it. No questions asked—just play. It should allow seeking and volume control with the keyboard and mouse. It should work with an infrared remote controller and also play television. It should play DVDs and VCDs too. MPlayer can do all of this and much, much more. MPlayer also is an award-winning, mature, open-source pro¬ gram that still is actively in development. Perhaps one day in the not-too-distant future, support for DVD menus, color subtitles, picture in picture video, MIDI and audio effects plugins like the ones for SoX will be added. MPlayer is admirably stable for the job it does. However, it does crash under certain circumstances, such as with certain video drivers. It is a mature application that has no parallel. MEncoder, its companion video encoder program, does a much better job than FFMPEG in transcoding videos, although it is a bit difficult to use and learn. Now, let's take a look at MPlayer's magic. The following command plays a stream URL after resampling it to 48,000Hz and combining channels into left-right stereo: $ mplayer -at lavcresample=48000,hrtf http://mp3.streampower.be/radiol-mid.mp3' The following command grabs the same URL stream and dumps it to a file named stream.aac: $ mplayer -dumpstream -dumpfile stream.aac ^-softvol -softvol-max 2000 -at ^lavcresample=48000,volnorm=2 :0.5 http://mp3.streampower.be/radiol-mid.mp3' You can open the stream file from another terminal window with this command: $ mplayer stream.aac Then what happens? The first instance of MPlayer continues to dump the network stream to a file and the second plays it for you— time-shifted Internet radio. Cool, eh? Most of the switches are not necessary to accomplish this, but they show MPlayer's ability to use the Linux command line so elegantly. The -softvol and -softvol-max switches invoke the software volume control feature of MPlayer. It reduces the signal-to-noise ratio, but it can amplify the signal to very high levels. The volnorm=2:0.5 filter invokes the volume normalization audio filter. The first argument, 2, specifies that several samples are to be used to smooth the volume variations. The 0.5 sets the maximum amplitude to which you want the volume normalized. As you can see, MPlayer provides a high level of customization. The other options on the command line should not be difficult to decipher. Playlists The following is a command similar to the first one above. In this case, however, you specify a playlist URL. Unlike the above command, this one may not work for you, depending on whether the file happens to be available from the SHOUTcast site when you try it: $ mplayer -at lavcresample=48000,hrtf -playlist ’http://www.shoutcast.com/sbin/shoutcast-playlist.pls?rn **-=1025&f i le=f i lename .pis’ The -playlist option is used only with stream URLs that have a .pis extension in the stream. You can create your own playlists in a variety of ways. Here is one way to do it with the find command: $ find /home/girish/music -name "*mp3 ^-or -name "*ogg" > -/playlist.txt Now, fire up MPlayer with: $ mplayer -playlist -/playlist.txt The following variant shuffles the list and plays songs in random order without repeating songs: $ mplayer -shuffle -playlist -/playlist.txt Naturally, MPlayer is a media player. You don't have to limit your¬ self to audio files. You could add any MPlayer-playable media file into the mix, including videos, movies, television, radio and, of course, Internet streams. In addition to this simple line-based playlist format, MPlayer also has excellent support for ASX, M3U and other popular playlist formats. Special Effects There is good support for audio effects, and the karaoke effect especially gets interesting with certain songs. It is not perfect, but you can attenuate the voice in a song a great deal. Use the following command to activate karaoke mode: $ mplayer -af karaoke song.mp3 MPlayer also has a ten-octave band equalizer. The following command ignores the middle frequency bands and amplifies the frequencies around 31.25Hz by 7dB, 62.5Hz by 8dB, 125Hz by 5dB and all of the frequencies around 4, 8 and 16Hz are attenuated by 2dB: 84 | October 2007 www.linuxjournal.com $ mplayer -af equalizer=7:8:5:0:0:0:-2:-2:-2 video.mpg The following command gives a live effect to playback. Try it with songs that sound monotonous: $ mplayer -af extrastereo song.mp3 You can issue a command like the following to play the third song five times: $ mplayer songl.mp3 song2.ogg file.wav -loop 5 If you want to repeat the whole list five times, type the following instead: $ mplayer { songl.mp3 song2.ogg file.wav } -loop 5 You also can use -loop 0 to play something over and over again. Additionally, there are many audio effect plugins designed especially for multichannel and 3-D audio. If you want some really advanced audio effects, try the SoX Swiss Army knife. It is another command-line application that excels in professional audio effects. You can specify multiple audio filters on the command line and they are applied one after another in a chain. Figure 1. MPlayer Audio/Video Filter Chaining What if you like a certain audio filter chain and you want to save the resulting audio to a file? The following command saves the output of filtering to the file named filtered.wav rather than playing it: $ mplayer -ao pcm:file=fiItered.wav -channels 4 -af laveresample=48000,hrtf,pan=2:1:0:0:0.3:0.5:0.5:0:12 audio.ogg The lavcresample filter resamples the frequency of audio.ogg to 48,000Hz. The pan filter is a very powerful and sophisticated filter. It mixes the input audio channels into the specified output channels in various amplitudes. In this example, we use the -channels switch to specify four input channels. The first argument to pan is 2 to specify two output channels. In the first pair of arguments after that, the 1:0 specifies the amplitude of the first input channel that is fed into the two output channels. It goes into the left channel with an ASA COMPUTERS Want your business to be more productive? Tho ASA Servers powered by tho Inlol XoodT Processor provide the quality and dependability to keep up with your growing business. Hardware Systems for the Open Source Community-Sine e 1989 (Imux. l-mwliiat 3 , MotBSD, Op$nB$D, fitrimiK. MS. Htc 1U UUaadcrest/Clovertawn Storage Server Starts at - $1,741 p i u LMjni r n } 10B G 3 7 MQZ FB[ , ((i Gupporti 16 QD FI .. / - 1XZ&DOB lilswap * ‘ A par | HA I A II bt A ITB3lurau0 iriELathd. Max. - 3TB. 11J iDunl rarn S03Q CPU (IHr 1) E.lr> - ? CPU 1GB &07MQZ FBDIMMs Installed. mo I MM 3ATA-II Dm we iristallul RAID *: ri .it. ol In r 2X1&.-100.'1000 LAM onboard 2U W ao dc re st/C lave rtawn Storage Server StaiK at - $3Jf 1 - *ITD Gtorage instatlad. Max - 1 2TI 3U Dual cara 5050 CPU. - 1QD 66 7MQZ TDDlMMs Initallet - Gu pports 1 SOD TDDI MM - IS port SATA-19 RAID controller. - 1 0X200Q 5 hlswati 3ATA-II ivo - 3X1 0/1 00/1 COO LAN onboard - aoow h 1 PS ; irliita lla U 3U Woodcresf/Crovertawri Storage Server Starts at - $3 r 791 - 4TD Gtoraga inxlallcridl Max- 1 2TD. J - 3U Dual uuiq OQOQ CPU. tal/; -1QD 667MGZ FDDIMMh installed. H' - SLippurlE 1 tdC-il-S F-HE3IMM 1 HX JbtlH H hhwap KA 1A II llnunx in station 1 li r i. ■ r I I-iA I All K Alll ■ :■ ■ ■ il ■ ■ I m 2X1 CV1 00/1 GOO LAM o aboard. MOnw hT r> fi KK 5U Woodciest/Cl overt own Storage Server Starts at - $ t,6? 1 6TD Gloragc iniEalltd. Mbk- I STD 5U Dual core 50 50 CPU, 4GB iifl7HGZ FBDIHM: Installed $uppnrla 1 F&pIMM V+X240O& hhw«p KATA II [^rivni i JA pari SAEA-II KAII1 2X10/100/1000 LAM onboard. 03 Ow Rod PS. mr &U Vyaadcrest/Ctevertawn Storage server Starts at - $ 11,191 ■ 10TB Sturo -cjh in cl a Hail. Majt — 30TB SU Dual core 50 50 CPU. VXbObtl mslallnrt itiS 007MCIZ , Suppaili 37 I 5 H r-Ht)IMM ■ ‘10XZSDQB h m swap 3ATA-II Drives inslallcrU - 2X12 Pori SATA-I I Mullilana RAID oonl rolla 1X10 Pori G ATA-I I Mullilana RAID controlla - 2X10/100/1000 LAM onboard. ■ IK 50 W Kod Pa. All sysrsms Installed and lesred with user's choice of Linux distribution (tree). ASA Collocation—$75 per mown 2354 Calle Del Mundo, S a nta Clara, CA 95Q 54 www,asacomputers,com Email: sales@asacomputers.com P: t-800-REAL-PCS I FAX: 408-654 2910 Xeon inside' IntulE 1 , Ink'll Kpoev”, Intel Im-idto-tt, Inlets Itdmumtffl find the Iniu-l Inside^# 1 'iscfP lire IrLuJ^riiikrk^ or registered trademarks, of Intel Corporation or its suttsirtijrie^ in I he United Stales and other countries. Prites and availability subject to change without notice. Not rttfHJftsibte feu typograpliksl errors. INDEPTH 1 amplification factor of 1. The second input channel goes into the right channel with an amplification factor of 0.3 (0:0.3), and the third input channel is divided equally into both output channels (0.5:0.5). The fourth channel goes into the right channel with an amplification factor of 12. You can use the following command to re-encode the WAV file to Ogg: $ oggenc -q 9 filtered.wav The Command Subsystem MPlayer has a very rich input command processing subsystem that can be manipulated with the keyboard, mouse, joystick or URC remote control. You also can customize several keyboard keys to invoke MPlayer's controls. The following provides the list of keys that can be configured. The Figure 2. MPlayer with Basic OSD Support defaults are very sensible, and you may not need to change them: Figure 3. MPlayer OSD Menu $ mplayer -input keylist You also can find MPlayer's controls for seeking, volume control, brightness correction and other things with the following command: $ mplayer -input cmdlist The slave.txt file that comes with MPlayer's documentation explains how to customize the input.conf and menu.conf files that come with the MPlayer package in your Linux distribution. You also can have MPlayer accept input commands from a FIFO file (FIFO stands for First In, First Out). This comes in handy when it is reading the media from the standard input: $ mkfifo /tmp/fifo $ cat playlist.txt | mplayer -input ile:/tmp/fifo -cache 8192 -playlist - Typing the following mutes and executes it again with the 86 | October 2007 www.linuxjournal.com audio unmuted: $ echo 'mute' > /tmp/fifo You can use the mouse for the usual seek operations. The mouse wheel is configured by default to seek files in both directions. On-Screen Display Apropos of input methods, MPlayer has excellent support for on-screen display (OSD). I normally use only the superb scalable TTF fonts. These are not available on the MPlayer Web site. The following command displays a timer: $ mplayer -osdlevel 3 -font ^/home/gi rish/.ttffonts/Comicbd . ttf video.avi See Figure 2 for a sample of this display. You can create a custom menu to browse using the keyboard with the OSD facility. It even has a console where you can enter MPlayer slave commands. As you can see, the scalability of the OSD fonts does not break with the video scale filter. Here is how to create this kind of configuration. First, set up most everything in the config file. Here is my -/.mplayer/config file: # Write your default config options here! # Use Matrox driver by default. vo=sdl font=/home/girish/.ttffonts/comi cbd.ttf vf=hue,eq,screenshot #menu-startup=yes menu=yes subfont-autoscale=3 subfont-osd-scale=8 subfont-text-seale=8 subpos=50 spuaa=4 osdlevel=3 # I love doing headstand (Sirsasana) while watching videos. #f1ip=yes # Decode/encode multiple files from PNG, # start with mf://filemask mf=type=png:fps=25 # Eerie negative images are cool. #vf=eq2=1.0:-0.8 INDEPTH Do you take "the computer doesn f t do that" as a personal challenge? So do we. LINUX JOURNAL Since 1994:The Original Monthly Magazine of the Linux Community The input configuration and menu configuration are stored sep¬ arately in input.conf and menu.conf, respectively. The menu.conf file needs to have a section that looks something like this: You need this line in input.conf in order to tell MPlayer what event invokes the menu. The setting here invokes the menu if you click the left-mouse button: MOUSE_BTN0 menu main Custom Subtitles Not surprisingly, MPlayer understands close to 12 subtitle formats, and it has its own MPSub format too. The options for subtitle dis¬ play are the richest I have seen. You can display subtitles in any size, any position on the video, move them dynamically with the keyboard, adjust the delay, change the transparency, format them into multiple lines and so on. Here is the most basic usage of the file subtitles.txt: FORMAT=TIME # first number : wait this much after # previous subtitle disappeared # second number : display the current # subtitle for this many seconds 2 3 What is going on? 4 3 How are you doing? 8 3 You are wrong! 0 3 A long long, time ago... in a galaxy far away... 0 3 Naboo was under an attack. 0 200 I don't understand this. Subscribe today at uvww.linuxjournal.com Tell MPlayer to use this file with a com¬ mand like this: $ mplayer -sub subtitles.txt **-font ~/.ttffonts/Verdana.ttf video.avi This next command dumps the subtitles file into the srt format into the file dumpsrt.sub in the current directory: $ mplayer -sub subtitles.txt video.avi ^-dumpsrtsub You can take a quick look at all subtitles in the file by pressing the Y and G keys. Of course, you can specify multiple subtitle files, and you can switch between them. Create Screenshots Want to take screenshots with MPlayer? It's easy. Here's a sample command to use when you start to play a video: $ mplayer -vf screenshot video.avi Press S when you want to take a screen- shot. If you want a screenshot every five sec¬ onds, try the following command: $ mplayer -vo png -vf screenshot -sstep 5 ^video . avi What if you want to take a screenshot of every frame? Set MPlayer to accept slave commands with a FIFO, and type these commands: $ mkfifo /tmp/fifo $ mplayer -input file:/tmp/fifo video.mpg $ echo ’screenshot 1’ > /tmp/fifo Toggle the screenshot process with the following command while the video is playing: $ echo ’screenshot 1’ > /tmp/fifo You might want to use the -vf spp,scale=1024:768 switch to get full-screen screenshots. Even More Power There's much more MPlayer can do. You can encode image files into a video and extract frames into image files with MPlayer. You also can watch analog television with the tv:// option and watch DVB channels with the dvb:// option. It supports a wide variety of streaming protocols, including RTP, RTSP, MMS, SDP and LIVE5555 streaming. Discover the Power Yourself The following command lists the available filters: $ mplayer -af help The man page and MPlayer's HTML docu¬ mentation have more thorough descriptions of its options. Typing: $ mplayer -vo help lists the compiled video output drivers. You can play an arbitrary audio file with the video using: $ mplayer video.mpg -audiofile audio.aac Of course, MPlayer can play a wide variety of audio and video media files. The following commands list them: $ mplayer -vo help $ mplayer -ao help Try using the -audio-demuxer switch along with -rawaudio. I hope this gets you started in discovering the awesome power of MPlayer. Enjoy your multimedia experience!* Girish Venkatachalam is an open-source hacker deeply interested in UNIX. In his free time, he likes to cook vegetarian dishes and actually eat them. He can be contacted atgirish1729@gmail.com. Resources MPlayer: mplayerhq.hu MPlayer HTML Documentation: www.mplayerhq.hu/DOCS/ HTML-single/en/MPlayer.html MPlayer Tips: freshmeat.net/articles/view/747 SoX: sox.sourceforge.net TS-7800 High-End Performance with Embedded Ruggedness $269 qty 1 $229 W 100 500 MHz ARM9 » New unbrickable design- 3x faster » Backward compatible w/TS-72xx * Low power - 4W at 5V » 128MB DDR RAM * 512MB high-speed onboard Flash * 12K LUT user-programmable FPGA * Internal PCI Bus, PC/104 connector * 2 USB 2.0 480 Mbps * Gigabit ethernet » 2 SD sockets » 10 serial ports » IIOGPIO » 5 10-bit ADC » 2 SATA ports » Sleep mode uses 200 microamps » Boots Linux in < 2 seconds » Linux 2.6 and Debian by default Design your solution with one of our engineers * Over 20 years in business * Never discontinued a product * Engineers on Tech Support a Open Source Vision * Custom configurations and designs w/ excellent pricing and turn-around time * Most products stocked and available for next day shipping See our website for options, peripherals and x86 SBCs 2 Technologic SYSTEMS We use our stuff. visit our TS-7P00 pnwftmrl wnhsitfi at www.embeddedARM.com -(480) 83/-S200- INDEPTH J FS-Cache and FUSE for Media Playback QoS Use FS-Cache to remove fluctuating performance issues from media playback, ben martin The FS-Cache Project works with network filesystems like NFS to maintain a local on-disk cache of network files. The project is split into a kernel module (fscache) and a daemon (cachefilesd), which help to maintain the disk cache. The local on-disk cache is maintained under a directory on a local filesystem. For example, the /var/fscache directory on the ext3 filesystem /var. The filesystem containing the fscache directory must have the ability to use Extended Attributes (EAs). Such filesystems are quite common and include ext3 and xfs. Early Fedora Core 6 kernel RPMs contained the fscache kernel module. Unfortunately, around version 2.6.18-1.2868.fc6 of the updated kernels, the module was no longer included. Fedora 7 kernels do not include the kernel module. Hopefully in the future, this module will be available again in standard Fedora kernels. The Fedora Core 6 update kernel 2.6.20-1,2948.fc6 has an FS-Cache patch included, but it does not include the kernel module. Patches are available for the Linux kernel for the FS-Cache kernel module (see Resources). The cachefilesd daemon communicates with the kernel module using either a file in /proc (/proc/fs/cachefiles) or a device file (/dev/cachefiles). Version 0.7 and earlier versions of cachefilesd could communicate only via the proc file; Version 0.8 also can use the device file if it is available with fallback to the proc file. Setting Up cachefilesd For Fedora Core 6 and Fedora 7, there is a cachefilesd RPM. Installation without package management should be fairly easy also, as the daemon mainly consists of a single executable and a configura¬ tion file (/etc/cachefilesd.conf). The two main things that need to be set up in the configuration file are the path of the directory to use under which to store the filesystem cache and options for controlling how much space is acceptable to use on the filesystem containing the cache directory. You also can supply a tag for the cache if you want to have multiple local disk caches operating at the same time. The space constraints all have acceptable defaults, so the cache directory is the only configuration option you need to pay attention to. Make sure that this directory is acceptable for storing caches and that it exists prior to trying to start cachefilesd. For a media PC, using a directory on a Flash memory card or on a RAM disk is a good option. Because the cache directory must have extended attributes, and your tmpfs might not include support for them, you may have to cre¬ ate an ext3 filesystem in a single file inside your tmpfs filesystem and then use the embedded ext3 filesystem for the cachefilesd path. The ext3 filesystem inside the single file happily will support extended attributes. Because the whole ext3 filesystem is in a single file on a RAM disk, it will not cause distracting disk 10 on the media PC. The fstab entry in Listing 1 sets up both a 64MB of RAM filesystem and the mountpoint for the embedded ext3 filesystem. The commands shown in Listing 2 set up the embedded ext3 filesystem. As the cache.ext3fs filesystem exists only in RAM, you have to add these commands to your /etc/rc.local or a suitable boot-time script to set up the cache directory after a reboot. This script has to be called before cachefilesd is started. Leaving cachefilesd out of your standard init run-level startups and starting it manually from the rc.local just after you set up the cache.ext3fs embedded filesystem is a good solution. If the cache directory is on a persistent filesystem, such as /var, set cachefilesd to start automatically, as shown in Listing 3. The space constraints in the configuration file are used to set the per¬ centage of available blocks and files on the filesystem containing the local cache directory that should be used. For each of these two resource types, there are three thresholds: cull-off, cull-start and cache-off. When the cull-off limit is reached, no culling of the disk cache is performed, and when the cull-start limit is reached, culling of the disk cache begins. For example, for the disk block type constraint, setting cull-off at 20% and cull-start at 10% means that as long as the disk has more than 20% free blocks, nothing from the cache will be culled. Once the disk reaches 10% free blocks, cache culling begins to free up some space. If the disk man¬ ages to get to the cache-off limit (say, 5%), the cache will be disabled until there is more cache-off space available again. Listing 1. Using a RAM Disk to Store the Local fscache On-Disk Cache tmpfs /var/fscache tmpfs size=64m,user,user_xattr 0 0 /var/fscache/cache.ext3fs /var/fscache/cache ^ext3 loop=/dev/loopl,user_xattr,noauto 0 0 Listing 2. Setting Up the Embedded ext3 Filesystem # mount /var/fscache # cd /var/fscache # dd if=/dev/zero of=cache.ext3fs \ bs=1024 count=65536 # mkfs.ext3 -F cache.ext3fs # mount cache.ext3fs Listing 3. Starting the cachefilesd Daemon and Setting It to Auto-Start Next Boot $ su -1 # service cachefilesd start # chkconfig cachefilesd on 90 | October 2007 www.linuxjournal.com The configuration options are prefixed with b for block type constraint and f for the files-available constraint. The configuration file has a slightly different naming method from that used above. For block constraints, the cull-off limit is called brun. For cull-start, the limit is called bcull, and cache-off is called bstop. Modifying Mounts To turn on FS-Cache for a mountpoint, you have to pass it the fsc mount option. I noticed that I had to enable FS-Cache for all mount- points for a given NFS server, or FS-Cache would not maintain its cache. This should not be much of an issue for a machine being used as a media PC, because you likely will not mind having all NFS mounts from the file server being cached. The fstab entry shown in Listing 4 includes the fsc option. Adding this fsc option to all mountpoint references to fileserver:/... will enable FS-Cache. Listing 4. fstab Entry for Mounting an NFS Directory on the Fileserver with FS-Cache fileserver:/too /too nfs bg,intr,soft,fsc 0 0 Preemptive Caching At this stage, FS-Cache will store a local cache copy of files, or part thereof, which is read off the file server. What we really want is to have data from files we are viewing on the media PC to be read ahead into the local disk cache. To get information into the local disk cache, we can use a FUSE module as a shim between the NFS mountpoint and the application viewing the media. With FUSE, you can write a filesystem as an appli¬ cation in the user address space and access it through the Linux kernel just like any other filesystem. To keep things simple, I refer to the application that provides a FUSE filesystem simply as a FUSE module. The FUSE filesystem should take the path to the NFS filesystem we want to cache (the delegate) and a mountpoint where the FUSE filesystem is exposed by the kernel. For example, if we have a /Ho me Movies NFS mount where we store all our digital home movies, the FUSE module might be mounted on /CacheHomeMovies and will take the path /HomeMovies as the delegate path. When /CacheHomeMovies is read, the FUSE module will read the delegate (/HomeMovies) and show the same directory contents. When the file /CacheHomeMovies/venice-2001 .dv is read, the FUSE module reads the information from /HomeMovies/venice-2001 .dv and returns that. Effectively, /CacheHomeMovies will appear just the same as /HomeMovies to an application. At this stage, we have not gained anything over using /HomeMovies directly. However, in the read(2) implementation of the FUSE module, we could just as easily ask the delegate (/HomeMovies) to read in what the application requested and the next 4MB of data. The FUSE module could just throw away that extra information. The mere act of the FUSE module reading the 4MB of data will trigger FS-Cache to read it over the network and store it in the local disk cache. The main advantage of using FUSE is to allow caching to work properly when the video playback is sought. The main disadvantage is the extra local copying where the FUSE module asks for more information than is returned to the video player. This can be mitigated by having the FUSE module request only the extra information every now and then—for example, reading ahead only when 2MB of data has been consumed by the application. For optimal performance, the read-ahead should happen either in a separate thread of control in the FUSE module and use readahead(2) or asynchronous 10, so that the video playback application is not blocked waiting for a large read-ahead request to complete. The FUSE Shim The fuselagefs package is a C++ wrapper for FUSE. It includes the Delegatefs superclass, which provides support for implementing FUSE modules that take a delegate filesystem and add some additional functionality. The Delegatefs is a perfect starting point for writing simple shim filesystems like the above nfs-readahead FUSE module. The read-ahead algorithm is designed to read 8MB using asynchronous 10, and when the first 4MB of that is shown to the application using the FUSE filesystem, it then reads another 8MB using asynchronous 10. So there should be, at worst, 4MB of cached data always available to the FUSE module. The C++ class to implement the shim is about 70 lines of code, as shown in Listing 5. Two offsets are declared to keep track of what the file offset was in the previous call to fs_read() and at what offset we should launch another asynchronous read-ahead call. The aio_buffer_sz www.logicsupply.com m supply Mini-ITX Systems for Embedded Linux Applications solid-state reliability ◄ x86 platform compatible ◄ fanless, low-acoustic operation ◄ Discover the Advantage of Mini-ITX. www.linuxjournal.com October 2007 | 91 1 is declared constant as an enum so it can use it to declare the size of aio_buffer. When aio_consume_window bytes of the aio_buffer are shown to the application using the FUSE filesystem, another read ahead is performed. If debug_readahread_aio is true, the FUSE module explic¬ itly waits for the asynchronous read-ahead to finish before returning. This is handy when debugging to ensure that the return value of the asynchronous 10 is valid. A non-illustrative example would have some callback report if an asynchronous 10 operation has failed. The main job of schedule_readahread_aio() is possibly to execute a sin¬ gle asynchronous read-ahead call. It updates m_startNextAIOOffset to tell Listing 5. Entire FUSE Shim C++ Class #include m_startNextAIOOffset = offset using namespace Fuselage; using namespace Fuselage::Helpers; + aio_consume_window; if( debug_readahread_aio ) #include { #include while ( (retval = aio_error( &arg ) ) == EINPROGRESS ) #include #include {} cerr << "aio_return():" using namespace std; << aio_return( &arg ) << endl; class CustomFilesystem } public Delegatefs { / } typedef Delegatefs _Base; off_t m_oldOffset; public: off_t m_startNextAI00ffset; CustomFi lesystemQ enum { _Base(), aio_buffer_sz = 8 * 1024 * 1024, m_startNextAIOOffset( 0 ), aio_consume_window = aio_buffer_sz / 2, m_oldOffset( -1 ) debug_readahread_aio = false \ • { \ s > char aio_buffer[ aio_buffer_sz ]; / virtual int fs_read( const char *path, void schedule_readahread_aio( int fd, char *buf, size_t size, off_t offset, bool forceNewReadAHead ) off_t offset, struct fuse_file_info *fi) { ff( m_startNextAI00ffset <= offset { cerr << "fs_read() offset:" << offset || forceNewReadAHead ) { << " sz:" << size << endl; int fd = fi->fh; cerr << "Starting an async read request" << " at offset:" << offset << endl; bool forceNewReadAHead = false; if( (offset - size) != m_oldOffset ) ssize_t retval; ssize_t nbytes; struct aiocb arg; bzero( &arg, sizeof (struct aiocb)); arg.aio_fildes = fd; arg.aio_offset = offset; arg.aio_buf = (void *) aio_buffer; arg.aio_nbytes = aio_buffer_sz; arg.aio_sigevent.sigev_notify = SIGEV_NONE; { cerr << "possible seekQ between read()s!" << endl; forceNewReadAHead = true; aio_cancel( fd, 0 ); } schedule_readahread_aio( fd, offset, forceNewReadAHead ); m_oldOffset = offset; retval = aio_read( &arg ); if( retval < 0 ) return _Base::fs_read( path, buf, size, offset, fi ); cerr << "error starting aio request!" << endl; } }; 92 | October 2007 www.linuxjournal.com Listing 6. Makefile for the FUSE Shim nfs-fuse-readahead-shim: nfs-fuse-readahead-shim.cpp g++ nfs-fuse-readahead-shim.cpp \ -o nfs-fuse-readahead-shim \ -D_FILE_0FFSET_BITS=64 -Ifuselagefs itself when the next asynchronous read-ahead call should be made. The forceNewReadAHead parameter allows the caller to force a new asyn¬ chronous read-ahead for cases such as when a seek has been performed. The fs_read() method is a virtual method from Delegatefs. It has similar semantics to the pread(2) system call. Data should be read into a buffer of a given size at a nominated offset. The fs_read() method is called by FUSE indirectly. The main logic of our fs_read() is to check whether the given offset is in a logical sequence from the last read call. If the offset is not sequential from the last byte returned from the previ¬ ous read(), fs_read() will force schedule_readahread_aio() to perform another read ahead. schedule_readahread_aio() is always called from fs_read() so it can handle the sliding asynchronous read-ahead window. As Delegatefs knows how to read bytes from the Delegate filesystem, we then can simply return by calling up to the base class. The remainder of nfs-fuse-readahead-shim.cpp is taken up by parsing command-line options, and instead of returning from main(), it calls the main method of Listing 7. simpleread.cpp Reads from argv[1] at a Nominated usee Rate in argv[2] #include #include #include #include #include #include using namespace std; int main( int arge, char** argv ) { cerr << "opening argv[l]:" << argv[l] << endl; long offset = 0; int fd = open( argv[l], 0_RD0NLY ); unsigned long usee = 10000; if( arge > 2 ) { stringstream ss; ss << argv[2]; ss >> usee; } cerr << "using delay of usee:" << usee << endl; const int bufsz = 4096; char buf[ bufsz ] ; bool error = false; a Delegatefs through an instance of the Custom Filesystem class. The shim is compiled with the Makefile shown in Listing 6. Taking It for a Spin A simple application that reads from a given file at a predetermined rate can verify that the cache is being populated as expected, as shown in Listing 7. There isn't a great deal of error checking going on, but things that would cause grief, such as failed read()s, are reported to the console. The application repeatedly reads 4KB chunks at a time from a nominated file and throws away the result. Every 256KB status is reported, so that the application can be closed knowing roughly what byte of the file was last read. As shown in Listing 8, we first clean out the cache directory and restart cachefilesd. Then, the NFS share is mounted and the FUSE shim run against it to create a /Cache-HomeMovies directory. The FUSE executable is told to remain in the foreground, which stops FUSE from running it as a daemon, allowing standard output and standard error of the FUSE filesystem to be displayed. We use bash to put the nfs-fuse-readahead-shim into the background (though still having its standard outputs redirected into a capture file) and run the simpleread for a little more than 500KB of data. Then, both the simpleread and nfs-fuse-readahead-shim are stopped to investigate whether the cache has been populated as expected. The simpleread was stopped after reading only a little more than half a while( true ) { ssize_t re = read( fd, buf, bufsz ); if( re > 0 ) { if( error ) { cerr << "reading resumed" << endl; } error = false; offset += rc; } else if( rc == 0 ) { cerr << "end of file" << endl; exit(0); } else { error = true; cerr << "read error:" << errno << " at offset:" << offset << endl; } usleep( usee ); if( offset % (1024*256) == 0 ) cerr << "offset:" << offset << endl; } return 0; } www.linuxjournal.com October 2007 | 93 INDEPTH 1 megabyte. However, the FUSE module has an asynchronous 10 call at the start, requesting 8MB of data be sent to it. Poking around in /var/fscache for a file with the same size as venice-2001 .dv should reveal the cache file. Listing 8. Running simpleread against the FUSE Shim # rm -rf /var/fscache/* # /etc/init.d/cachefilesd restart # mount fileserver:/HomeMovies /HomeMovies -o fsc # nfs-fuse-readahead-shim --fuse-forground \ -u /HomeMovies /Cached-HomeMovies \ >|/tmp/nfs-fuse-out 2>&1 \ & # simpleread /Cached-HomeMovies/venice-2001.dv 1000 using delay of usec:1000 offset:262144 offset:524288 A C Comparing the first 8MB of this cache file to the version on the NFS share should show that the first 8MB is identical. Note that the local cached file is read first to make sure that the subsequent use of the NFS share does not populate the cache file before it is read. This is shown in Listing 9. Listing 9. Checking That the Cache Has Read the First 8MB # cd /var/fscache # 11 -R - 1 root root 8 00M Jun i 0 0 2:19 Ek0...000000 # dd if=./path/to/Ek0...000000 \ of=/tmp/8mb bs=1024 count=8192 # dd if=/HomeMovies/venice-2001.dv \ of=/tmp/8mb.real bs=1024 count=8192 # diff /tmp/8mb.real /tmp/8mb # # fg A C # Wrap-Up One restriction on FS-Cache is that it will not cache files opened with 0_DIRECT or for writing. Ultra Dense, Powerful, Reliable... Datacenter Management Simplified! 15” Deep, 2-Xeon/Opteron or P4 (w/RAID) options Customized Solutions for... Linux, BSD, W2K High Performance Networking Solutions • Data Center Management • Application Clustering • Network and Storage Engines Rackmount Server Products • 1U Starting at $499: C3-lGHz, LAN, 256MB, 20GB IDE • 2U with 16 Blades, Fast Deployment & more... iron SYSTEMS Iron Systems, Inc. 540 Dado Street, San Jose, CA 95131 www.ironsystems.com Call: 1 -800-921 -IRON By taking advantage of the kernel FS-Cache code, the FUSE module to handle read-ahead can be very simple to create. The Delegatefs C++ FUSE base class allows one to implement additional features very easily when applications perform 10. The FUSE nfs-fuse-readahead-shim module is started just as shown in Listing 8 and when the —fuse-forground option is not passed, nfs-fuse-readahead-shim runs silently as a daemon. ■ Ben Martin has been working on filesystems for more than ten years. He is currently working toward a PhD combining Semantic Filesystems with Formal Concept Analysis to improve human-filesystem interaction. Resources Filesystem in USErspace (FUSE): fuse.sourceforge.net FS-Cache: people.redhat.com/ ~dhowells/cachefs and people.redhat.com/~steved/fscache fuselagefs and Delegatefs: sourceforge.net/project/showfiles.php? group_id=16036&package_id=225200 NFS and fscache Kernel Patches: people.redhat.com/~dhowells/ cachefs/patches 94 | October 2007 www.linuxjournal.com LISA '07 offers the most in-depth, practical system administration training available! 6 DAYS OF TRAINING BY EXPERTS IN THEIR FIELDS, INCLUDING: • Tom Limoncelli on Time Manage¬ ment for System Administrators 3-DAY TECHNICAL PROGRAM, INCLUDING: Keynote by John Strassner, Motorola FeUowand Vice President , Autonomic Networking and Communications, Motorola Research Labs, on Autonomic Administration: HAL 9000 Meets Gene Roddenberry • Steve VanDevender on High- Capacity Email System Design Invited Talks by industry leaders discuss important and timely topics, including: • ytleen Frisch on Administering Linux in Production Environments • Faisal N. Jawdat on Introduction to Ruby, Rails, and Capistrano • Bruce Moxon, NetApp, "A Service-Oriented Data Grid: Beyond Storage Virtualization” • Erik Nygren, Akamai Technologies, “Experiences with Scalable Network Operations at Akamai” • Kenneth G. Brill, Uptime Institute, “The Economic Meltdown of Moore's Law” Sponsored Register by October 19 and save! www.usenix.org/lisa07/lj sage Refereed Papers, Hit the Ground Running Track, Guru Is In Sessions, Vendor Exhibition, Workshops, BoFs, WiPs, and more! NOVEMBER 11-16, 2007 /var/opinion More Power to Linux New digs means an opportunity for Linuxization of the home. Nick Petreley, Editor in Chief What a month. I'm thinking of changing my name to Nicholas Almighty in hope God will send me plans for an ark. This has been one of the rainiest years on record for south central Texas. The rain hasn't bothered me too much because the weather is still warm, but it seems as if anything more than a drizzle causes power and cable interruptions in my little section of San Antonio. I have potent power-surge protectors and a UPS (that stopped working recently, unfortunately), but the frequent power surges from having the power drop out repeatedly eventually fried my server, and then fried the RAID cage in the server after I fixed it. I didn't think the latter was possible, but I have no other explanation for why, immediately after one of the latest power outages, the RAID cage stopped recognizing one of the drives. I guess electronics is electronics, and anything can blow. My replacement cage arrived literally as I was typing this, so I should have a working server again soon. I'm hoping that by the time you read this, I will have moved to a house in a small town just outside San Antonio. I don't know if the new location will solve my power problems or make them worse, but I do know it means I have to give up cable, and along with it, Roadrunner. The only Internet access in this area is dial-up, satellite and wireless. I opted for wireless broadband, although I use the term "broad¬ band" loosely, as we're talking about a maxi¬ mum of about 800Kbps download speed. When I signed up, the fellow remarked that I was about the fourth Linux user he'd encoun¬ tered this month. He confessed that he's not very familiar with Linux, although his company runs Linux on some servers. He said there's a noticeable increase in Linux usage among his customers. That didn't surprise me, but his next comment did. He said he figures Linux is gaining some popu¬ larity because it's no longer a command-line- only operating system. Perhaps I shouldn't be surprised. Most computer users can remain blissfully ignorant of how long Linux has had superior graphical desktops, because they have something they consider to be good enough. Most of us are of the opinion that alternatives like Windows aren't good enough, because we've been feasting on steak for quite some time. We have a point of reference. They don't. I have a feeling that Linux evangelism doesn't work very well for spreading the word that Linux is a viable desktop. The fact that Dell now preloads Linux on desktop machines works much better. People visit Dell for a variety of reasons. It gives Linux visible credibility to see it as an option for Dell machines. In fact, I can't help but won¬ der if Dell's decision to preload Linux had something to do with this fellow hearing more about Linux users. One of the dedications in a Linux book I wrote credits Bill Gates and Steve Ballmer for promoting Linux by making Windows more undesirable with every release. Look for a big influx of Linux users if Microsoft turns a recent patent idea into reality. Microsoft wants to add insult to injury by making Office as undesirable as Windows. The idea is to make its office applications (Word, Excel and so on) examine your documents and collect data about you, so that Microsoft can sell targeted advertising. I'm not sure where Microsoft expects to place the smart ads. Perhaps they'll appear in a window within its word processor and spreadsheet. If so, this puts a whole new spin on an old "dippy" talking paper-clip joke. Now, you'll get a cus¬ tom ad like, "It looks like you're composing a suicide letter. May we interest you in Gillette razor blades? They're on special this week at your local Walgreens." I, for one, say go for it Microsoft. A rush to OpenOffice.org or any other alternative office suite would be nothing but good news for Linux. Moving to a permanent residence means I can start thinking about a Linux-based home. The house is too small to do anything really fancy, but I can build a simple multime¬ dia box that also controls a number of house functions like lights and the thermostat. Ideally, I'd like to use voice commands to control some things. It's too bad that so little work seems to be ongoing for Linux voice recognition. A cursory search turned up a list of projects, but most of them haven't been touched in years. Sphinx 4 (cmusphinx.org) looks like the only decent active project. If you know of any others, please share. I'm also thinking of setting up a tiny recording studio in the garage for myself and my kids. The Rosegarden music editor and midi sequencer may not be as powerful as some of the Windows commercial products, but it's remarkably feature-rich. It certainly has enough power for our meager talents. Regardless, I'm looking forward to whatever Linuxization I can do to my new home. Wish me luck, but if you say, "more power to you", I prefer it to be steady state, no surges. ■ Nicholas Petreley is Editor in Chief of Linux Journal and a former programmer, teacher, analyst and consultant who has been working with and writing about Linux for more than ten years. 96 | October 2007 www.linuxjournal.com "Fanatical Support" saved me from my own mistake." "Not long ago, I reformatted one of our servers. Not until I was driving home did I learn that I brought our entire site down in the process. I called my guy at Rackspace and he said, 'We're already on it.' By the time I pulled in the driveway, my site was back up. Now that's Fanatical Support." Keeping little mistakes from causing big problems is one definition of Fanatical Support. What will yours be? Watch Russ's story at www.rackspace.com/fanatical 1-888-571-8976 MANAGED I HOSTING WhisperStation™ Cool... Fast... Silent! Hear Yourself Think Again! For 64-bit HPC, Gaming and Graphic Design Applications Originally designed for a group of power hungry, demanding engineers in the automotive industry, WhisperStation™ incorporates two dual core AMD Opteron™ or Intel® EM64T™ processors, ultra-quiet fans and power supplies, plus internal sound-proofing that produce a powerful, but silent, computational platform. The WhisperStation™ comes standard with 2 GB high speed memory, an NVIDIA e-GeForce or Quadra PCI Express graphics adapter, and 20" LCD display. It can be configured to your exact hardware specification with any Linux distribution. RAID is also available. WhisperStation™ will also make a system administrator very happy, when used as a master node for a Microway cluster! Visit www.microway.com for more technical information. Experience the “Sound of Silence”. Call our technical sales team at508-746-7341 and design your personalized WhisperStation ™ today. +Microway Technology you can count on ” M