Monday, September 13, 2004

More Testing

Without using the switches (-mx and -oss), I was able to use 91,080 KB before running out of memory (weka.jarjava.lang.OutOfMemoryError). I ran Weka using the following:
C:\Program Files\Weka-3-4>java -jar

With using the switches (-mx and -oss), I was able to use 123,804 KB before again running out of memory (weka.jarjava.lang.OutOfMemoryError). I ran Weka using the following:
C:\Program Files\Weka-3-4>java -mx100000000 -oss100000000 -jar

Even though I was able to use up to 123,804 KB (~ 121 MB) before running out of memory, it isn't sufficient to produce the results that I would like (By the way, I've been running on a machine with 1GB of RAM).

I have attempted various methods in order to stay within memory limits. As I'm somewhat unsure of what I'm mining for, I have selected attributes that seem merely seem most interesting to me. For instance, I removed every column except for the mission and state. I then ran j48 and it succeeded! The tree visualization, however, wasn't very impressive since it on had these two attributes.

It has been discouraging to be constantly running out of memory.

I clustered the complete dataset and found nothing very interesting. The KMeans clustering algorithm didn't require much memory.

I created a complete E-R Diagram of the LDSM database. I'd like to meet with Dr. Giraud-Carrier, an experienced data-miner, and talk about the diagram and determine what more (if anything) would be interesting to run on the data. I'd like to eliminate useless attribute columns so that I can achieve some interesting results before exhausting the memory.

1 comment:

Fred said...
This comment has been removed by a blog administrator.