Export data from Postgres

To export data from Postgres to an output file of your choice can be done by following the simple steps below:

1. Start psql with the database that you'd like to export from...

$ psql [DATABASE]

2. Toggle the output mode to unaligned (\a toggles between unaligned and aligned output mode)

=# \a

3. Turn "tuples only" off (\t toggles between tuples on and off)

=# \t

4. Set the output file (replace [FILE] with what you'd like to call your output file). It will send all query results to the file or |pipe.

=# \o [FILE]

5. Run whatever query you'd like to send to the output file. For example,

=# SELECT * FROM [TABLE];


In summary:

\a
\t
\o /tmp/outputfile.txt
SELECT ......
\o

Personalized Marketing

Personalized marketing as a four phase process:

1. identifying potential customers

2. determining their needs and their lifetime value to the company

3. interact with customers so as to learn about them

4. customize products, services, and communications to individual customers

From Wikipedia, “Personalized marketing,” (Cited: Peppers, D. and Rogers, M. 1993)

Dimensionality Reduction Notes

Principal Components Analysis (PCA)

How do you choose how many and which eigenvalues/eigenvectors to use?

Kaiser Criterion
This says to retain only factors with eigenvalues greater than 1. In other words, if a factor does not extract at least as much as the equivalent of one original variable then it is discarded. This criterion is named after Kaiser as he proposed it in 1960. It seems used quite frequently.

The Scree Test
This is a graphical test used to decide how many factors to keep. To perform this test, first, plot the eigenvalues in decreasing order. Next, Cattell suggests to find the place where the smooth decrease of eigenvalues appears to level off (to the right) similar geological scree (loose rock debris at the bottom of a rocky slope).

Here are some other useful terms and definitions from the dictionary:

Multicollinearity refers to linear inter-correlation among variables. Simply put, if nominally "different" measures actually quantify the same phenomenon to a significant degree -- i.e., wherein the variables are accorded different names and perhaps employ different numeric measurement scales but correlate highly with each other -- they are redundant.

K-Means Clustering

Basic Algorithm
1. Choose k cluster centers at random
2. Assign each point to nearest cluster center
3. Compute the new cluster centers based on the assigned points
4. Repeat until cluster centers converge

Shortcomings
Finds local minima
The random placement of cluster centers affects the outcome

Here is a nice K-Means Demo

Taking Genealogy to the Common Person

Subversion

Handy Links for Subversion:
Download Subversion Client
http://subversion.tigris.org/faq.html

Google and data

Here is an interesting article about Google and all their data.

GEDCOM file format information

The following links discuss the how the GEDCOM format is defined:
Cyndi's List
The GEDCOM Standard Release 5.5
GEDCOM: The Next Generation

Networked Data File Types

Here are reference links to common network data file types used in Link Mining and Social Network Analysis:
Pajek Net File
UCINet DL Files and VNA

Machine Learning Topics

Particle Swarm Optimization
wikipedia
Swarm Intelligence

Ant Algorithms
ant colony optimization

Reinforcement Learning
wikipedia
Q-learning
Q-learning definition
Markov decision process

Computational Learning Theory
wikipedia
VC dimension
Principle of maximum entropy

Ensembles, Bagging and Boosting
Boosting

Meta-Learning
METAL KDD
Christophe Giraud-Carrier

HMMs
Hidden Markov model

Viral Marketing

Dr. Ralph F. Wilson suggests that Viral Marketing is comprised of the following components:
1. Gives away products or services
2. Provides for effortless transfer to others
3. Scales easily from small to very large
4. Exploits common motivations and behaviors
5. Utilizes existing communication networks
6. Takes advantage of others' resources
The effects of word-of-mouth, or viral marketing are motivations for utilizing the social network that customers belong in.

Customer Segmentation

Customer analysis helps a business better meet customer needs. Learning more about your customers is often benefited by intelligent segmentation. Customers can be segmented into a variety of groups. These segments can be based on behavioural, demographic, geographic, and psychographic variables. In fact customers can be segmented by any combination of these variables. Through viewing customers within such segments the problem of identifying and serving customers is simplifed. The knowledge provided by these segments is usually useful for determining actionable marketing tactics.

Stanford Data Mining Course

Stanford offers a nice Data Mining and Electronic Business course within the Statistics department. It looks like it covers many exciting aspects of the field.

What is Lift?

In data mining, "lift" is often used to measure model performance. Here is a link to an article that explains how it is used: DMReview article

IP Country Lookup Tool

Here is a link to a tool that I created to lookup the countries for all of the IP addresses in a mess of text.