mcgyver5: Technology

Tue, 27 Apr 2004 15:56:46 GMT

While every article about application development impresses upon us the importance of separating the data, logic and presentation layers, MS Excel gleefully mixes the three. While this gets sneers from most developers, it is precisely the reason that so many people use it and love it and get themselves into all kinds of trouble with it... and then ask me to help. So I was very excited to get my copy of Excel Hacks today. Among other things, it has a lot of tips for separating the data from the formulas and the data from the presentation, making a complex Excel spreadsheet more maintainable. Other things I was glad to see:

Chapter devoted to the SpreadsheetML, or the xml/xslt format used to describe spreadsheets. This facilitates generating spreadsheets using PHP and Java and other technologies that have nothing to do with Excel
Lots of hints for overcoming limits imposed by Excel: For example, override the number of undos allowed and override the limit to criteria for conditional formatting of cells.
Lots of easy to understand VBA code
Ways to recover data from corrupt workbooks
Long chapter on charts with some neato ideas for custom charts. I get asked a lot of questions about Excel charts and I never really learned more than basic charting skills.
And a question I have tried to solve several times, never getting a graceful solution: How to fill in empty cells with values
Solutions to pivot tables

I am working my way through each of the 100 hacks and in almost every one, I am saying to myself, "Gee, I didn't know excel could do that" In summary, the O'Reilly hacks series has produced some excellent books and this one is one of the best. It is clear, concise, error free, and doesn't assume you have the most recent version of Excel. It is full of nuggets such as the fact that Excel purposely doesn't recognize that the year 1900 was a leap year (link blames it all on Lotus!). Who knows, you might encounter that some day.

DNR Trout Stream Easement GIS Layer

Wed, 07 Apr 2004 21:23:59 GMT

My cube neighbor is creating a database of all the easements owned by the state along trout streams. It will soon be available in GIS format. That is, you can download the data and make pictures like this, which is the new data layed over arial photos. This particular selection is Lower Gavin Brook in Stockton, MN:

Tue, 06 Apr 2004 23:06:45 GMT

I wanted to remove a range of items from a list using the List.subList method. I tried the following:

	// al is defined previously as ArrayList al = new ArrayList();
        List l = al.subList(20,50);
        al.removeAll(l);

That got me the java.util.ConcurrentModificationException

The documentation says that a subList is dependent on the List that it is sublisting from. So, while a sublist is open, it is manipulating the parent list and seems to have a lock on it. Instead, you can create a brand new ArrayList and fill it with the subList, thus freeing up the sublist so that a future call can modify the original List:

	ArrayList sl = new ArrayList(al.subList(10,40));
    	al.removeAll(sl);

So why does

List = myList.subList(10,30);

get you a different animal than

ArrayList temp = new ArrayList(myList.subList(10,30)); ??

because in the first snippet, I haven't instantiated a new object, I just created a reference to an existing object. In the second snippet, I created a brand new object. That's what the word "new" gets me.

Compacting Radio DataFiles

Sun, 04 Apr 2004 12:42:40 GMT

I just compacted my data files in Radio. I noticed that I had all these huge files under the folder RadioUserland/datafiles weblogData.root was 16 Megs. After running compaction that file is just 2 Megs. aggregatorData.root went from 37 Megs to 400 kb

I also noticed when backing up that problems occur when you have radio open and you try to copy a 37 Meg file that radio is trying to use.

Mon, 29 Mar 2004 20:03:37 GMT

IntelliJ IDEA If you use IntelliJ 4.0 to check out projects from CVS make sure you click the "Change keyword substitution" to binary button in the CVS checkout procedure. Failure to do this could cause your project to fail to compile since the jar files will be corrupt.

Mon, 29 Mar 2004 16:19:57 GMT

IntelliJ IDEA, the Java development tool that I am liking more and more, uses log4J to build a log file that is stored as : USER_HOME\.IntelliJIdea\system\log\idea.log

(On my windows installation, USER_HOME is C:\Documents and Settings\timcguir )

This file has lots of information about the internal workings of IDEA and has error messages that explained to me why the IDEA GUI Builder wasn't working like I expected.

Fri, 26 Mar 2004 21:12:08 GMT

You can put a link tag on your web page that tells the visiting browser to start downloading another page while it is idle so that your browsing seems faster:

< link rel="prefetch" href="/images/big.jpeg" >

from 0xDECAFBAD

Static

Wed, 17 Mar 2004 23:55:17 GMT

1 exerting force by reason of weight alone without motion

2 showing little change

An early frustration in learning Java is the error: non-static method foo( ) cannot be referenced from a static context
Perhaps you are trying to call a simple method of your class from the main() method. The main method is a static context. The quick solution is to instatiate your object with MyObject mo = new MyObject right inside your main method. now you have an instance of MyObject
A static variable or method is going to persist across all the objects instantiated.
Static means that you lose the use of the 'this' keyword because the use of 'this' implies that you have an instantiation of an object.
Static means that it doesn't depend on the instantiation of an instance of the object. It still exists and can be called from an instance of the object, but it can also be called without creating the object.
The main() method is static because you have to call it to start a java program. That is, you call it before any objects are instantiated.

Consider the following class:

public class AddMe{
	
	static int num = 100
	
	static int add(int i){
		int sum = i+2;
		return sum;
	}
}

When I want to access the variable num, I can just say, myNum = AddMe.num and get the number 100.
I can also instantiate AddMe am = new AddMe() and then call myNum = am.num with the same effect.
Same with the add method. myNum = AddMe.add(2) sets myNum to 4 and so does myNum = am.add(2)
During my one week java class from Sun, I completed an exercise by using all static methods. I could then call the methods just like I made function calls in PHP or VB. My solution was used as an example of bending Java to act like a procedural language. Which, in case you need to be told, was frowned upon. It avoids all the Object Oriented concepts of Java. You don't even ever need to instantiate a class!

Tue, 16 Mar 2004 17:20:32 GMT

My Findings on HashMaps:

A Map is one type of Collection in Java. Collections are used to hold a bunch of objects. Different Collections let you look up your objects in different ways.
The special thing about Maps is that the way you look up your objects in a Map is with another object.
A Map is like an associative array in PHP. It is a container for key - value pairs.
To get values out of a Map, you iterate through the Map's keys and use the get() method to get the value associated with that key back out.
Map is the interface, Hashmap or TreeMap is the implementation of that interface, just like ArrayList is an implementation of List. So, to make a new HashMap, you would say Map myHashMap = new HashMap();
A HashMap uses the hashCode() method available to all Java objects to take the hashCode of your object so that it can quickly find it when you want it.
with a HashMap, your objects will not be sorted the way you expect
with a TreeMap, the objects are sorted in a way that makes sense.

JavaScript Disable Links

Mon, 15 Mar 2004 17:52:50 GMT

Here is the javascript function I used to disable the link that people often click more than once:

function disable_links(){

// this function disables all the links on the page

for(var i=0; i < document.links.length;i++)

{

document.links[i].onclick=function () { return false; }

document.links[i].title = "The exam has been submitted. Please wait for results page";

}

return true;

}

Mon, 15 Mar 2004 14:39:39 GMT

For two weeks I had this really frustrating problem:

When users submitted the online fisheries exam, sometimes they would get an error page while the results would be emailed with no problems. I kept testing it and never got the error. I wondered if the database was crapping out because of too many open connections, I wondered about browsers caching error pages.

Then I watched someone submit the exam. They were like, "it sometimes takes several clicks before it submits"..... click click click. It is second nature to me not to submit a form more than once, no matter how long it takes the server to process the form.

So, when testing, click all the buttons more than once, because it's a sure bet your users will.

In any case, there should be some code that handles this problem, but I didn't write the thing, I just inherited it.

new radio tools

Mon, 15 Mar 2004 13:16:13 GMT

Roger's Cadenhead, the author of the excellent Radio Userland Kickstart, has released the first version of his workbench.root

It has two scripts:

Workbench.viewCategories(), a script to display category links on your home page template and other pages.

Workbench.viewPostIndex(), a script to display links to your posts in reverse chronological order

ArrayLists

Sat, 13 Mar 2004 12:57:53 GMT

I had the hardest time using an ArrayList. I had an ArrayList full of Fish objects and getting them out with ArrayList.get() and wanting to do some Fish type operations such as printing out data about the Fish and comparing it to another Fish.

To get the third Fish out of the ArrayList it is in, I tried

currentFish = myArrayList.get(2);

The trick is that ArrayLists (and other containers) store all objects as Objects instead of as Fish or Dogs or Cats or Cars or Strings. This behavior happens for all types of java Collections. So when you call an object back out of an ArrayList, you have to cast it back into whatever object it is supposed to be

currentFish = (Fish)myArrayList.get(2);

Further confusion happens because every object has toString() over-ridden and automatically available so that when you think you are getting a string back from an ArrayList and want to print it out, it works fine.

I even had it print out the results of getClass() on the Object I was retrieving from myArrayList and even that told me it was a Fish! But still, it was being treated like an Object until the cast.

Thu, 11 Mar 2004 19:50:36 GMT

Added notes on using IntelliJ IDEA GUI builder.

Wed, 03 Mar 2004 00:39:06 GMT

To he who wrote the java programs I have to maintain:

I curse your name 10 times daily.
I curse you for having different stuff checked in to version control than is actually in production.
I curse you for inventing your own stupid inscrutable templating system.
I curse you for not leaving any comments in your code.
I curse you for leaving some parts of your application calling a test database and others calling a production database.
I curse you for naming methods f2, f3, f4, f5, f6, f7, f8, f9 etc but also for having a method named fl which in my editor is indistinguishable from f1
I curse you for the critical javascript functions called blah and blah2.
I curse you for leaving me a java class with 3000 lines.
I curse you for naming things ShowBean and MenuBean when they aren't really beans at all.

IntelliJ Rulez

Wed, 25 Feb 2004 04:36:52 GMT

I bonded with IntelliJ IDEA tonight. I really started to feel like I was holding the hammer the right way for the first time.

BANG! automatic try-catch blocks

BANG! rename one method and tell it to update your entire project

BANG! alt-insert to populate all your getters and setters for a bean

BANG! automatically generate import statements

BANG! ANT compiles your files and puts them in the right place

Lustron homes

Tue, 10 Feb 2004 17:27:06 GMT

Kate told me the other morning that she saw an interesting show on PBS at like 3:00 am about metal and ceramic prefab homes from the 1940s. I would have thought she was dreaming, but her story did not feature small animals chewing on her, so I knew it was real. It was the Lustron homes story. The story goes like this:

Housing shortage after WWII
Visionary inventor comes up with prefab houses that are cheap and last forever without maintenance.
People line up to buy them. Thousands are built and sold all over the country.
unknown forces conspire against the company.
Company goes bankrupt.
Lustrons live up to their promise. Their owners love them. There are seven of these in Minneapolis.

Sun, 01 Feb 2004 02:01:06 GMT

I like the PHP function mysql_insert_id( ) . I just found out about it. It is good for when you just inserted on one table and you want to insert the autogenerated primary key into another table. Almost serves as the equivalent of an Oracle sequence object, but not quite.

Works as follows

[ME]: Damn. I wonder how to find the user ID for the brand new user without doing another query?

[Narrator from Dukes of Hazzard]: "Well, son just try $user_id = mysql_insert_id(); "

Thu, 04 Dec 2003 02:06:43 GMT

According to Google,

B is for B'Tselem

C is for CNET

D is for D-Link

Check out the entire google alphabet:

from a blogger who turns out to be a Macalester Grad

and neither does my wireless network

Sun, 30 Nov 2003 16:30:57 GMT

I have a TRENDnet wireless USB network adaptor (TEW-USB) and a TRENDnet wireless base station. I know the base station works because my brother got connected to it last night. My wireless client does not work, however, and the documentation really is lacking. Can't find any troubleshooting information on-line or in the PDF that comes with the driver. TRENDnet online support does seem to be responsive, though, so have a problem ticket and I will wait for them to get back to me. Meanwhile, I have another box with pretty lights and a donut shaped electromagnetic radiation field encompassing my house and yard.

Send Notepad Cruising!

Sat, 29 Nov 2003 15:54:42 GMT

Help a guy who has been helping PHP programmers for free for a long while. Notepad hangs out at www.codewalkers.com answering people's questions for free. He wants to go on the PHPCruise and I daresay he deserves it. Codewalkers has set up a donation page for him and they have already raised half of his trip cost. Support the PHP community and thank an active participant!

The Underscore Hack

Mon, 24 Nov 2003 23:38:26 GMT

Well, cool. those crazy Internet Explorer stylesheet problems can be kind of resolved with this hack:

http://www.pixy.cz/blogg/clanky/cssunderscorehack/

November TCWEBDEV Meeting Notes

Fri, 07 Nov 2003 10:41:00 GMT

just geek talk, sweet beautiful geek talk.

The tcwebdev meeting last night was all about graphics, discussion led by Justin Heideman.

One up and coming graphics format is jpeg2000.
- It has a better compression algorithm than regular jpeg and the main advantage is smaller file sizes.
- Mozilla has a plug-in for it, but IE and photoshop and other programs will not recognize it yet.
- If you want to open a picture in jpeg2000 format, download the free irfanview software.
Finally learned why Fireworks likes to save images as png files, because saving them as jpeg over and over again increases the loss that the compression brings. PNG24 is lossless, so you don't have that problem
PDF:

A reason to buy Adobe Distiller is that it can compress PDFs smaller than other software.

Color

Suggestion to visit Target stores to take pictures and get good color combination ideas for web sites.
Or buy a $5.00 color guide book full of pallettes

Software Requirements Part I

Tue, 04 Nov 2003 13:31:43 GMT

The DNR Project Requirements Group meets for the first time today. The assigment for the first meeting was to read part one of "Software Requirements" by Karl Weigers, a requirements guru. Published by Microsoft Press.

The metrics should give pause to any developer. The costs of not doing requirements well are staggering. On average, rework consumes 30 - 50 percent of total costs and requirement errors account for 70 to 85 percent of reasons for rework
The first chapter gives the top reasons for requirements errors. The one that jumped out at me was "CREEP"
The author suggests using testing to constantly verify requirements. This can be verbal. You have a meeting with the customer and walk through what the software will do. It can also involve a rigorous verification process where a team of people look at your requirements documents and try to find errors.
After reading this, I know I want to create a project glossary so we can have a reference for all the crazy terms fisheries biologists throw around.
Establish a baseline document that the customer agrees to. This document serves to connect cost estimates, time estimates, requirements, basis for acceptance of completed project and a jumping off point for change management. The first thing to strive towards is a baseline for each of the following documents:
1. Vision and Scope statement "why are we building this?"
2. Use Case Document "Customers can make reservations"
3. Software Requirements Specification "The system shall send the customer an email verification of the reservation"
The book suggests creating user interface and technical prototypes, even a preliminary implementation. This seems to fly in the face of all the cautions in the book, "dont start coding until requirements are done" I question this.
Version control on the requirements documents. Book suggests using a requirements management tool. Our analyst has a home grown one that he has developed over his career and it seems effective.
hold facilitated elicitation workshops. **Shudder** We have tried several of these. Without the other stuff in place, they didn't bring us to where we needed to be. We didn't create usable requirements documents out of them and to do it again now may bring accusations that we are going over the same ground again.
Don't use sign off as a weapon. Sign off means the customer has agreed to your estimates and requirements. I guess this means don't wave the requirements in your customer's face and scream, "You already bought it, sucker!!!" Use it instead as a basis for change management
This section of the book leaves me wondering how estimates of time and money are connected to requirements. It says "Developers are in the best position to estimate costs" and then the next sentence says "many developers are not skilled estimators" so, how do we do it?
The book has a nice "bill of rights for software customers" followed by a "bill of responsibilities for software customers" "Make timely decisions" is one of these...
Finally, it has a list of good requirements characteristics. "Avoid lumping requirements into long narrative paragraphs. " bullets, bullets bullets.

A warning to ye...

Tue, 28 Oct 2003 16:47:51 GMT

I can't impress upon ye enough: set error_handling to E_ALL (verbose) when developing PHP. You will cut your errors in half. You can do this in three ways:

uncomment the following line in your php.ini file: error_reporting = E_ALL . This will affect all PHP scripts
add the line error_reporting = 2047 in an .htaccess file. This will affect only php files in or below the directory of the .htaccess file.
add a line of code at the start of every php page: error_reporting (E_ALL); This will affect only the files that you alter.

After you do this you may get a whole page of warnings about undeclared variables. This is a good thing, since it forces you to declare all variables and lets you know when you introduce new ones into the mix by mistake. You will at some point mis-type variable names. Without error reporting turned verbose, PHP accepts these mistakes as real variables:

$species_code = "WAE";

if($speces_code == "WAE"){ //do stuff}

The above code exhibits a bad feature of PHP; that you don't need to declare variables before using them. I introduced a variable called speces_code to the PHP script and since it is blank it is not equal to "WAE". It just skipped over this comparison without a blink!

Also, set error reporting to zero when in production so that visitors cannot peer into the inner workings of your code and directory structure when they encounter an error.