There are a number of measures of correlation for spikes of two neurons and for spikes at two successive time bins in one neuron: covariance, correlation coefficient, mutual information, and information-geometric measure in the log-linear model. It is desirable to have a measure that is not affected by change in the firing rates of neurons. We explain the superiority of the information-geometric measure from the point of view of geometry, by which the correlation and firing rates are separated orthogonally, that is, without correlation. We then analyze characteristics of other measures and show analytically how they are related to firing rates.