-Di

I have stumbled across a tricky point in the preprocessing of data. The most relevant post is probably

this of April 7. Rather than lecture, let me ask and answer some questions. The fundamental question is:
Can I inadvertently reduce the rank (the dimensionality) of the data matrix?
The answer is yes.

Suppose we have a data matrix of full rank. Suppose the matrix is a typical one, in having more rows than columns. That the matrix be of full rank is equivalent to: the columns are linearly independent.

There is a serious asymmetry here: not all of the rows are linearly independent.

We might as well imagine some specific numbers. Suppose we have 3 columns and 5 rows. Since we assume it is of full rank, the rank of the matrix is 3.

The term "centering" means to "make zero-mean." A centered column is one with zero mean = zero sum; a centered row is a row whose mean (hence sum) is zero..

What happens to the rank of the data matrix if we only row-center the data?
The rank decreases by 1.

Visualize the matrix as 3 column-vectors c1, c2, c3 of length 5. That the row sums (= row means) are zero says that c1 + c2 + c3 = 0, so the three columns are linearly dependent and the matrix is of rank 2.

What happens to the rank of the data matrix if we only column-center the data?
Nothing.

Visualize the matrix as 5 row-vectors r1, r2, r3, r4, r5 of length 3. We have more than 3 vectors of length 3, so they span a 3D space, but no larger. Adding the constraint r1 + r2 + r3 + r4 + r5 = 0 imposes, in general, no additional restriction. (If they had somehow spanned a 5D space, it would, but they can't.)

What happens to the rank of the data matrix if we only set the row sums to 1 (or 100), i.e. to a nonzero constant value?
Nothing.

All we’ve done is assert that r1 + r2 + r3 = 1, a constant vector. These 4 vectors are linearly dependent, but the original 3 are not.

But. It has another potential effect, probably undesirable.

If we were to use the resulting data matrix for a regression, we would find ourselves in trouble. We almost always want to adjoin a column of 1s to the data matrix, in order to include a constant term in the regression equation. We just got through saying that the combination r1, r2, r3, 1 is linearly dependent: the regression will explode with a singular matrix. Once we have set the row sums, we would have to drop a column of data in order to include a constant term. This is the same problem we face in regression when we define instrumental (dummy) variables.

What happens to the rank of the data matrix if we only set the column sums to 1 or some other nonzero constant value? (I’ve never seen it done, but why not, at least in principle?)
Nothing. (It even has no effect on regression.)

First Summary

So, there were 4 cases: whether we set row or column sum to a constant value, and whether that constant was zero or not. Of those 4 cases, only one (row sum = 0) reduces the rank of the data matrix.

What I was unclear about – or downright wrong about – on April 7 was in saying that we would lose rank by setting the row sums to 1 or 100 etc. Not so; none of the other 3 cases reduce the rank of the matrix.

Is that it?
No. All those questions were about one single operation. What if we combine operations?

Can't we center both the rows and the columns?
Yes. We saw
here on April 13 that we could doubly-center the data.

Does that mean we lose rank?
Yes, because we are centering the rows. Centering the columns is irrelevant.

Can we set the row sums to 1 and then center the columns?
Yes.

But something else will happen. Centering the columns is no longer irrelevant.

Didn’t we see that if the row sums were constant, and then we centered the data, the row sums went to zero?
Yes. It may seem weird, but it's true.

That’s an easy way to get doubly-centered data using Mathematica®. Scale the rows so they each sum to 1, then center the columns. Voila’, the row sums will change to zero.

Any time you start messing with row and column sums or means, remember the grand mean, too: the average of all the data. It is also both the average row mean, and the average column mean. If the columns all have zero mean, then the grand mean is zero. Then the mean of the rows (edit: i.e. of the row means) must be zero. The challenge is to show that the rows still have a common sum (edit: equivalently, a common mean); then that common sum mean must be the mean of the rows, i.e. zero.

If we set the row sums, say to 1, and then column-center the data, do we lose rank?
Yes.

Column-centering (or standardizing) the data after setting the row sums will reset the row sums to zero, and that loses rank. If it is appropriate, however, that the data be column-centered, so be it. I do not believe that reduction of rank is sufficient grounds for not centering the data.

Second summary

The apparently innocuous act of column-centering can lose rank, if we apply it to nonzero-constant-row-sum data. We can do it, and sometimes we should do it, but we should expect to lose rank when we do. But we’re losing rank because we first set the row sums, even though to a nonzero constant.

I had previously said I would favor doubly-centered data. Now I am inclined not to do so without cause, simply because it reduces the rank of the matrix.

Can I inadvertently reduce the rank (the dimensionality) of the data matrix?
Hell yes. In particular, we might find that we effectively centered the data without realizing it.

Suppose we set the row sums of our data to 1, say, and then we decide that rather than do an SVD of the new matrix, we will do an eigendecomposition of the covariance (or correlation) matrix.

Oops. Computing either the covariance or correlation matrix is tantamount to column-centering the data.

here on March 10.

Goodbye full rank.

Third summary

Suppose we have data for which a constant nonzero row sum is appropriate. Suppose we choose to do an eigendecomposition of the covariance (or correlation) matrix.

Then we lose rank. Constructing either of those smaller matrices implicitly centers the columns, and that in turn implicitly changes the row sums of the data to zero. Even if we never explicitly compute the transformed data.

The clearest path in such a case might well be as follows. First set the row sums to 1 or whatever nonzero. Then column-center or standardize the data. We will see the rank drop by 1 at that point. Then do an SVD. The only advantage to doing it this way, rather than via the covariance or correlation matrices, is that we see when we lose full rank.

If we live in a universe where data is supposed to be centered, or where the covariance or correlation matrix is appropriate, then we will lose rank if, in addition, the data had constant row sums.

Final summary

0. We could set row or column sums to zero or to a nonzero constant.
1. Setting row sums to zero is the only one of the four cases that will reduce rank.
2. Column-centering applied to data with constant row sums leaves us still with constant row sums but the sums have been changed to zero. This gives us (1) and loss of rank.
3. An accidental way to achieve (2) and thus (1) is to analyze the covariance (or correlation) matrix of constant-row-sum data.

It's not a problem if you know what happened and why.

Är du som jag äckligt duktig på att slösa stora summor på sms och samtal? Bli medlem du också. Kostar inte ett skvatt, man behöver typ bara gå in på lite hemsidor osv. (inte köpa något). Goodie för plånboken.

-------------------

Btw. Amy Winehouse verkar vara oerhört poppis ftm, apropå hennes kollektion av makeup och accessoarer möjligen. Verkligen en tjej på ropet.

Olá querida Sassá;

Ora, o que é isso. É sempre um prazer poder ajudar!

Mande sua foto 3X4, uma de corpo inteiro, preencha o formulário e vamos estar enviando sua candidatura ao posto.

(não comenta com ninguém, mas ouvi falar que uma tal de Rihanna tava no pé dele. Mas com um nome desses, deve ser baranga.) Tenha fé!

beijos,meliga!

Portsmouth - Johnson misses! Man United retain the Community Shield!

Here's the full report on the winning start-

http://www.espnstar.com/football/premier-league/news/detail/item101071/Man-Utd-beat-Pompey-3-1-on-pens/

Hey, EPL is about to begin, start scheduling yourself, book your slots to see some great kick-starters in the first 2-3 weeks.

Era mais um final de semana como outro qualquer na capital paranaense. Estava frio, cinza e uma chuva fina e gelada caia ininterruptamente. Era dia de sofá, a preguiça assim determinou.

- Eu escolho o filme hoje.

- De jeito nenhum! Eu escolho o filme hoje.

- Certo, vamos fazer iguais pessoas adultas fariam.

- Hum.. cara ou coroa?

- Não! Isso é para franguinhos e não somos franguinhos. Quer dizer.. você é..

- A quem ta chamando de franguinha, hein? Hein? HEIN?

- Então vamos lá.. eu tenho um desafio pra você. Se você conseguir vencer, eu assisto o que você quiser.. nem que isso envolva aqueles malditos filmes do Harry Potter.

- Fechado!

- Então.. é o seguinte... Faça uma matriz transposta da identidade, ache seu determinante e a sua jacobiana. Chamando o determinante da transposta de "x" e sendo o mesmo um espaço percorrido num tempo "t", onde "t" seja a jacobiana da matriz. Integre os resultados em função de "t" achando assim a velocidade. Decomponha a velocidade no espaço R3 em seus autovetores originais i, j e k. Após isso, calcule o produto interno do vetor velocidade encontrado com a matriz identidade prévia e me diga o resultado. Fácil assim. Tem 15 minutos!

- Não preciso de 15 minutos, já sei a resposta.

- Já?

- Sim. Jacobiana se fez Jacobiana pela dissecação do seu vetor externo, para, assim, atrair vetores convergente numa soma enlouquecida de operações de vetores que, então, se tornariam internos. Entendo, portanto, que a transposta da matriz é, na verdade, a inversão da identidade da matriz, algo que muito me chocou!!! Pois ao saber que o mesmo, ou mesma, queria decompor o escalar da velocidade em várias posições dos seus autovetores no espaço R3. Fazendo para isso uso, só posso pensar eu, das mais obscuras relações matemáticas. Portanto, me sinto obrigada a ser determinante com a Matriz e sua nova identidade transposta e inversa e devo alertá-lo da impossibilidade da multiplicação de matrizes de raízes diferentes. Mas esclarecendo que se pode, ainda, integrar o "x" bem ao tempo "t", e, ao integrá-lo, tornar-se-á uma constante em sua vida e assim juntos viverão felizes para sempre, mas sem que ninguém derive, pois a derivada de uma constante nada mais é do que zero!!! E então?

- Sou seu fã, sabia?- diz entre risos - É difícil achar uma outra pessoa com tanto talento para ser excêntrica.

- Uiiii..Que medooo!

- Medo? Por quê?

- Não tenho certeza se quero ter fãs.. Lembra o que aconteceu com o Lennon?

- Ah! Mas eu não te mataria.. Não pelo menos não como fizeram com o Lennon..Você precisa de uma morte digna de você. Algo grandioso. Tinha que ser igual a do Kennedy, assim, na frente de milhões!! A do Lennon nem filmada foi...

- Eh, vendo por esse ângulo...não quero só escrever: "saio da vida pra entrar pra historia"..quero dizer em alto e bom som para as multidões que me amavam..

- Que te amavam não!! Que te amam e amarão, pois você nunca será esquecida. Estará para sempre no coração de todos os que um dia tiveram a felicidade de te conhecer. Será eterna, pois seu nome nunca morrerá. Será passado de geração para geração, na palavra escrita e falada, pois todos lembrarão do tempo em que o mundo tinha você...

- Ehh..e todos te odiarão..e o pior.. você vai usar aquelas roupas cinzas de prisão e bem.. você fica horrível de cinza.

- Bem, na verdade eu sempre fico bonitinho, mas vou ter um uniforme diferente: preto. Pra todos saberem quem foi o causador da maior desgraça da terra. Pra saberem quem destruiu o sonho de milhões. Claro que escreverei um livro sobre minha, ou melhor, nossa vida, que será transformado em filme e ganharei milhões.

- Milhões?

- Sim, até serei entrevistado no programa do Jô, no Late Show, etc. todos querendo entender o porque. Então direi: "Porque antes dela ser do mundo, ela havia sido apenas minha e vocês tiraram isso de mim. Então a tirei de vocês. Espero que gostem da sensação, seus malditos, malditos, todos vocês!!!".

- Não sabia que era tão possessivo assim comigo.

- Não sou... só um sentimento de posse doentiamente controlado, o resto é cena.. Você sabe que sou um puta de um ator né! Na verdade vou fazer tudo isso pela fama...

- Que isso.. nós sabemos que a culpa irá te corroer e você viverá pra sempre atormentado com a minha presença..

- Ahh! Mas eu já vivo atormentado com sua presença em dias normais..

- Como é que é?

- Err.. nada.. quis dizer que você escolhe o filme.. e aí.. Harry Potter?

This time is Manchester United and Portsmouth, for complete details, do visit this section on ESPNSTAR.

English Premier League kicks off soon, get ready, book your days and be ready as this season is going to be really exciting one!

Ausbildung, Fortbildung, Weiterbildung, Studium ... wie auch immer. Ich wünschte mir eine noch größere Durchlässigkeit in unserer bibliothekarischen Berufswelt für Menschen, die sich nicht so ohne weiteres einordnen lassen (wollen). Und die noch Ziele haben, ohne an ihren vergangenen Entscheidungen bzw. Möglichkeiten scheitern zu müssen.

Admitting it to my GP scares me.  It goes on my records, for a start.  Along with my depression and all the associated stuff.  And I am scared that she will judge me, as if anyone could judge me more harshly than I do myself!  There's a voice in my head telling me I'm greedy and disgusting, and it's not an eating disorder, I'm just a vile pig.  And so on.  And it's been there for years.

And I have to ask for help, which I find hard.  How do I explain in the allotted 5 minutes that I am out of control?  That it's no longer about the weight, I don't care what I fucking look like, I just want to feel comfortable in my own skin?  I want to feel safe around food, not want to eat everything that passes in front of my face.  That joke about 'I'm on a sea food diet - I see food and I eat it' hasn't been funny for a long time, because that's what my life is like.

Admitting it to my GP scares me.  It goes on my records, for a start.  Along with my depression and all the associated stuff.  And I am scared that she will judge me, as if anyone could judge me more harshly than I do myself!  There's a voice in my head telling me I'm greedy and disgusting, and it's not an eating disorder, I'm just a vile pig.  And so on.  And it's been there for years.

And I have to ask for help, which I find hard.  How do I explain in the allotted 5 minutes that I am out of control?  That it's no longer about the weight, I don't care what I fucking look like, I just want to feel comfortable in my own skin?  I want to feel safe around food, not want to eat everything that passes in front of my face.  That joke about 'I'm on a sea food diet - I see food and I eat it' hasn't been funny for a long time, because that's what my life is like.

First of all, he expects that in most cases one will neither standardize nor even center the data X. We can do his computations as an SVD of X, or an eigendecomposition of $latex X^{T}X$ or of $latex XX^T$ – but because the data isn't even centered, $latex X^{T}X$ and $latex XX^T$ are not remotely covariance matrices. For this reason, I assert that preprocessing is a separate issue.

Nevertheless, the underlying mathematics is the same: get either an SVD of X or an eigendecomposition of $latex X^{T}X$ and/or of $latex XX^T$. But what do we do to X first? That's a separate question.

Second, he is not primarily interested in eliminating "small" eigenvalues or singular values: he is interested in eliminating "experimental error", i.e. "noise". Although I worked thru his chapter 4 on estimating noise, I have not discussed it: his main interest is in deciding when x and $latex \hat{x}$ are "close enough", and without a real-world application, and without a more rigorous treatment, I’d rather pass for now. I’ll come back to his error stuff, however, if I ever find myself anywhere else looking at error estimation in PCA / FA.

In addition to omitting chapter 4, I stopped after chapter 5. What he has in common with Harman and Jolliffe is a lot of references to the literature. I didn't see anything after chapter 5 that I could confirm the computations of.

Third, he doesn't much use the usual vocabulary of scores and loadings, although he does use the subscript "load" for his $latex \hat{X}$ in contrast to the subscript "basic" for his matrix X of successful test vectors x. (I sometimes think that he uses subscripts for emphasis rather than to distinguish entities, but I’m probably exaggerating.) In any case, I decided that the customary vocabulary was secondary to the mathematics: find the eigenvector matrix or matrices.

Fourth, he has no graphical techniques; he provides none of the graphs we came to expect in Harman, Jolliffe, and Davis. Such graphs do, in fact, have a place in chemistry; the Brereton "Chemometrics" – which I have not yet discussed – has a few. We will not do much with it: their internet data is available only to owners of the book, so I can compute to my heart's content, but I can't very well publish the data and you can't very well follow along without it. But I will see if there's anything I need to say about it.

Fifth, his notion of target testing (including using it to fill in missing values) is a whole new world, a brave new world, and I like it. I think I did it more simply than he did, but I was just cleaning up the math.

I do wonder. Can target testing be used in PCA "rotations", i.e. change of basis? For handling multicollinearity in OLS? To tell us we should have centered the data? To tell us to subtract a constant? I don't know yet. I'll keep my eyes open.

I learned a lot from Malinowski about using the available tools. Davis taught me to use the SVD, Malinowski got me comfortable with using both the full and the cut-down SVDs. Not to mention using the u and v bases, and constructing the hat matrix.

OTOH, there is at least one thing we did along with Malinowski that the other authors did not do. They might have, but they did not: reconstitute the data matrix using the reduced set of eigenvalues or singular values.

The classical techniques generally just list the eigenvectors v1, possible weighted by the w0 (equivalently, by the square roots of the eigenvalues). But in principle, they could have computed D1. In practice, they usually got no closer than describing the new correlation matrix or variances.

When we start with the SVD

$latex D = u\ w\ v^T$

and replace the smallest singular value in w by 0 (and call the new matrix w0), we can reconstitute the data D as

$latex D1 = u\ w_0\ v^T$

("Reconstitute" is intended to convey that impression that D1 is not the real data, not fresh orange juice. )

I want to show you more detail of the difference between D and D1. I’ve been a little vague about it.

Recall example 5 with noise. Here’s the data matrix:

$latex D = \left(\begin{array}{lll} 1.9 & 3.2 & 3.9 \\ 1 & -0.2 & -1 \\ 4 & 4.9 & 6.1 \\ 3 & 1.9 & 1.1 \\ 6 & 6.9 & 7.9\end{array}\right)$

The singular values were $latex \{16.194,\ 2.41991,\ 0.238533\}$

We interpret 3 nonzero singular values to mean that the matrix D is technically of rank 3; we know we can go further, however, and say that D differs from a matrix of rank 2 by its smallest singular value, namely .238533.

(I could go so far as to say the matrix D is of rank 2.238533, but perhaps it's better to leave it at "differs from a matrix of rank 2 by .238533." After all, if the smallest singular value were 10, the matrix would differ from a matrix of rank 2 by 10, and I don't want to say that it's of rank 12 = 2+10.)

We replaced the smallest singular value (0.238533) by 0, and reconstituted the data, calling it D1.

$latex D1 = \left(\begin{array}{lll} 1.95859 & 3.05258 & 3.98415 \\ 0.992857 & -0.182028 & -1.01026 \\ 3.95401 & 5.0157 & 6.03396 \\ 3.02136 & 1.84625 & 1.13068 \\ 6.0016 & 6.89597 & 7.9023\end{array}\right)$

What is the difference between D and D1? D1 is of rank 2, and the difference is supposed to be 0.238533.

Just how do we compute that difference? That’s the question.

The appropriate "norm" – appropriate because it gives this answer – is called the Frobenius norm, and it's pretty simple: pretend the matrix is a vector, and compute its Euclidean norm (2-norm), namely the square root of the sum of squares.

Here we are. The element-by-element differences between D and D1 are:

$latex e1 = \left(\begin{array}{lll} -0.0585929 & 0.147418 & -0.0841454 \\ 0.00714334 & -0.0179725 & 0.0102586 \\ 0.0459858 & -0.115699 & 0.0660403 \\ -0.0213625 & 0.0537476 & -0.0306787 \\ -0.00160247 & 0.00403179 & -0.00230131\end{array}\right)$

Now take the square root of the sum of the squares of the "components".

In case you're working thru this with me, the squares of those numbers are

$latex \left(\begin{array}{lll} 0.00343313 & 0.0217322 & 0.00708044 \\ 0.0000510273 & 0.000323011 & 0.000105238 \\ 0.0021147 & 0.0133863 & 0.00436132 \\ 0.000456356 & 0.00288881 & 0.000941184 \\ 2.56792E^{-6} & 0.0000162553 & 5.29604E^{-6} \end{array}\right)$

and the sum of them is 0.0568979, and the square root of that is 0.238533.

That number should seem familiar: it's exactly the singular value that we set to zero; it's exactly what it ought to be.

By computing the difference e1, and then the square root of the sum of squares, I explicitly computed the difference D - D1 and confirmed that it was equal to the smallest singular value of D. In practice, of course, there is no need to compute the sum-of-squares, because we already know it.

I’ll close that by saying that if we set two singular values to zero, the difference between the original (D) and the reconstituted (D1) is the square root of the sum of squares of the two singular values. What’s going on is that the norm of D is the same as the norm of w, and the norm of D1 is the same as the norm of w0; so the difference between D and D1 is the difference between w and w0.

I think we're done here.

Oh, let me be clear. I’m glad I own Malinowski (ah, the book). If you're going to be doing PCA / FA in the physical sciences, you probably want it, too. If you're going to be doing PCA / FA in chemistry, don’t even think about not buying it. Just my opinion.

When Bowyer subsequently returned to the side for their UEFA Cup fixtures following his suspension, it was just an example of Newcastle's lack of morals - at least at a managerial level - as has been a decision that has once again allowed the club to make the headlines for all the wrong reasons.

Joey Barton, released from prison just a week ago, has been welcomed back to the Newcastle United squad by manager Kevin Keegan.

Keegan claims that Barton's behaviour prior to his sentence was exemplary and that as such, he deserved a second chance.

"I told him that as long as he did things right while he was with us and started to rebuild the confidence of everyone around him – which he had to do – I would stick by him," the Newcastle Utd manager told The Times on Monday, " people have opinions and you’ve got to respect those, but mine is to give him another chance," he said.

This is the same Joey Barton who's second, third and fourth chance came at Manchester City, when - while on tour with the club in Thailand - Barton attacked a 15-year-old Everton fan. This 'second chance' came after he had stubbed out a cigar in a team-mates eye the previous year. And then of course there was his horrific attack on his Manchester City team mate Ousmane Dabo.

So while Kevin Keegan claims that he is giving Barton a second chance - it will in fact have been his fifth chance - although argueably his first chance at his new club.

Newcastle do have a long history in showing cowardice in the face of an opportunity to do the right thing. Duncan Ferguson's general aggression and his astounding ability to subtley elbow opposition players in the face - an action that resulted in Gregg Berhalter of Crystal Palace suffering a severely broken jaw - was often a point of blissful ignorance.

They also chose not to take action when their chairman, Fred Sheppherd, was secretly recorded making disparaging comments about Newcastle's women and the club's own fans.

Their decision not to take action following Bowyer's and Dyer's fracas in the middle of the St James' Park pitch showed a new level of toothless-ness - to then follow this up with a reluctance to serve Joey Barton with anything other than a warm, welcoming hug shows the club in perhaps the worst light yet.

Joey Barton will return to the club - earning a ridiculous wage every week - when what he really deserves is the opportunity never to play football again. The FA will ban him for 15 games - which is far off the full time ban that the player really deserves.

I had thought that eq. 1.136 3.136 was a typo, but I no longer think so.

If what I’m about to do is what Malinowski was suggesting, then the credit is his; in any case, the mathematics looks good to me, and if it's not what he suggested, it is nevertheless my reaction to his work.

the data and its SVD

The cleanest way to make my case is to do a simulation. The following is the data from my example 5. It is of rank 2. In keeping with my latest notation, this is denoted by a "D", specifically D0 (rather than "X").

$latex D0 = \left(\begin{array}{lll} 2 & 3 & 4 \\ 1 & 0 & -1 \\ 4 & 5 & 6 \\ 3 & 2 & 1 \\ 6 & 7 & 8\end{array}\right)$

Let me add some noise to it.

Here's randomly generated and rounded noise...

$latex \left(\begin{array}{lll} -0.1 & 0.2 & -0.1 \\ 0 & -0.2 & 0 \\ 0 & -0.1 & 0.1 \\ 0 & -0.1 & 0.1 \\ 0 & -0.1 & -0.1\end{array}\right)$

which i add to D0 to get the final data matrix for this example. In keeping with my latest notation, that data matrix will be called D.

$latex D = \left(\begin{array}{lll} 1.9 & 3.2 & 3.9 \\ 1 & -0.2 & -1 \\ 4 & 4.9 & 6.1 \\ 3 & 1.9 & 1.1 \\ 6 & 6.9 & 7.9\end{array}\right)$

That is what we know, not D0. We can talk about D0 because this is a simulation, but in practice D0 is unknowable: we are trying to estimate it.

Get the Singular Value Decomposition of D, $latex D = u\ w\ v^T\ $. Here's w:

$latex w = \left(\begin{array}{lll} 16.194 & 0. & 0. \\ 0. & 2.41991 & 0. \\ 0. & 0. & 0.238533 \\ 0. & 0. & 0. \\ 0. & 0. & 0.\end{array}\right)$

Set the 3rd singular value to zero. We are assuming that it is a measurement of the noise – which it is, by design. (There will still be some noise left over.)

We could write either

$latex w0 = \left(\begin{array}{lll} 16.194 & 0 & 0 \\ 0 & 2.41991 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & 0\end{array}\right)$

or

$latex w1 = \left(\begin{array}{ll} 16.194 & 0. \\ 0. & 2.41991\end{array}\right)$

For w1, we need the cut-down u and v. Because w1 is 2x2, u1 is the first 2 columns of u, v1 is the first 2 columns of v. Then compute the "reconstituted data" $latex D1 = u1\ w1\ v1^T\ $, using the cut-down SVD. As usual, it is also true that $latex D1 = u\ w0\ v^T\ $, using the full SVD with only two singular values. Here is the reconstituted data D1:

$latex D1 = \left(\begin{array}{lll} 1.95859 & 3.05258 & 3.98415 \\ 0.992857 & -0.182028 & -1.01026 \\ 3.95401 & 5.0157 & 6.03396 \\ 3.02136 & 1.84625 & 1.13068 \\ 6.0016 & 6.89597 & 7.9023\end{array}\right)$

The difference between the data D and the reconstituted matrix is...

$latex D - D1 = \left(\begin{array}{lll} -0.0585929 & 0.147418 & -0.0841454 \\ 0.00714334 & -0.0179725 & 0.0102586 \\ 0.0459858 & -0.115699 & 0.0660403 \\ -0.0213625 & 0.0537476 & -0.0306787 \\ -0.00160247 & 0.00403179 & -0.00230131\end{array}\right)$

There is no way in the real world we would know the original matrix D0, but this is a simulation and here's the difference between D0 and D1:

$latex D0 - D1 = \left(\begin{array}{lll} 0.0414071 & -0.0525815 & 0.0158546 \\ 0.00714334 & 0.182028 & 0.0102586 \\ 0.0459858 & -0.0156993 & -0.0339597 \\ -0.0213625 & 0.153748 & -0.130679 \\ -0.00160247 & 0.104032 & 0.0976987\end{array}\right)$

The u and v computed from D (which has noise) are different from the u and v computed from D0 (which does not have noise); the two largest singular values (w1) are also different: I infer that we cannot eradicate the noise (D vs D0) just by eliminating the smallest singular value. If we could have, D1 would match D0.

In other words, the noise has caused D1 to differ from D0.

For reference, in case you are working along with me, here are the u and v matrices.

$latex u = \left(\begin{array}{lllll} -0.32924 & -0.320222 & -0.752815 & -0.205034 & -0.424605 \\ 0.0179826 & 0.577752 & 0.0917793 & -0.775085 & -0.238114 \\ -0.542068 & -0.155692 & 0.590837 & 0.11501 & -0.565338 \\ -0.201861 & 0.731055 & -0.27447 & 0.56307 & -0.180079 \\ -0.746118 & 0.0705562 & -0.0205889 & -0.1641 & 0.641075\end{array}\right)$

$latex v = \left(\begin{array}{lll} -0.485249 & 0.811213 & 0.326294 \\ -0.570892 & -0.0112849 & -0.820948 \\ -0.662281 & -0.584642 & 0.468591\end{array}\right)$

the RC decomposition, and Xhat from X

Moving on, do what we've done before: here's $latex R = u1\ w1$ and $latex C = v1^T\ $, so D1 = R C:

$latex R = \left(\begin{array}{ll} -5.33172 & -0.774911 \\ 0.291211 & 1.39811 \\ -8.77828 & -0.37676 \\ -3.26895 & 1.76909 \\ -12.0827 & 0.17074\end{array}\right)$

$latex C = \left(\begin{array}{lll} -0.485249 & -0.570892 & -0.662281 \\ 0.811213 & -0.0112849 & -0.584642\end{array}\right)$

Now, we test the following two X vectors (i.e. two columns of this X matrix):

$latex X = \left(\begin{array}{ll} 1 & 1 \\ 2 & -1 \\ 3 & 1 \\ 4 & -1 \\ 5 & 1\end{array}\right)$

You should assume that I tested thousands of other data records looking for likely candidates, and selected these two based on the following calculations.

To get $latex \hat{X}$ I want the hat matrix, $latex H = u1\ u1^T\ $:

$latex H = \left(\begin{array}{lllll} 0.210941 & -0.19093 & 0.228326 & -0.167639 & 0.223058 \\ -0.19093 & 0.334121 & -0.099699 & 0.418739 & 0.0273469 \\ 0.228326 & -0.099699 & 0.318078 & -0.0043966 & 0.393462 \\ -0.167639 & 0.418739 & -0.0043966 & 0.57519 & 0.202193 \\ 0.223058 & 0.0273469 & 0.393462 & 0.202193 & 0.56167\end{array}\right)$

I compute the $latex \hat{X} = H X\ $ (As I said, in the real world this computation is what selected the columns of X; the computation preceded the construction of X.)

$latex \hat{X} = \left(\begin{array}{ll} 0.958792 & 1.02089 \\ 1.98991 & -1.01614 \\ 2.93289 & 1.04396 \\ 3.96837 & -0.963772 \\ 5.07526 & 0.94865\end{array}\right)$

I compute the difference $latex \hat{X} - X\ $:

$latex \hat{X} - X = \left(\begin{array}{ll} -0.0412078 & 0.0208948 \\ -0.0100941 & -0.0161423 \\ -0.067115 & 0.0439618 \\ -0.0316295 & 0.0362281 \\ 0.0752581 & -0.0513498\end{array}\right)$

OK, they will do: the differences are small. We would have done that calculation for thousands of candidates. Now get $latex T = w1^{-1}\ u1^T\ X$ for the two candidates we chose.

$latex T = \left(\begin{array}{ll} -0.398759 & -0.0885231 \\ 1.50634 & -0.708358\end{array}\right)$

As before, we can confirm that the product R T

$latex R\ T = \left(\begin{array}{ll} 0.958792 & 1.02089 \\ 1.98991 & -1.01614 \\ 2.93289 & 1.04396 \\ 3.96837 & -0.963772 \\ 5.07526 & 0.94865\end{array}\right)$

is equal to $latex \hat{X}\ $.

Yhat from Y

Now, i'm going to change notation. Somebody has to do something! We write, as before

$latex D1 = R\ C = R\ T\ T^{-1}\ C = \hat{X}\ T^{-1}\ C\ $.

all i want to do is change is the name of $latex T^{-1}\ C\ $: call it $latex \hat{C}$ (instead of $latex \hat{Y}\ $). then we have

$latex D1 = \hat{X}\ \hat{C}\ $,

and $latex \hat{C}$ is

$latex \hat{C} = \left(\begin{array}{lll} 0.999354 & 0.970148 & 1.00377 \\ 0.979945 & 2.07897 & 2.95989\end{array}\right)$

Why change notation? Because i'm about to test some vectors y and construct an independent $latex \hat{Y}\ $. To put it another way, I desperately need something which is to Y as $latex \hat{X}$ is to X.

Duh, use $latex \hat{Y}\ $. (My first wish is that Malinowski had done this.)

Let $latex J = v1\ v1^T$ (This is the hat matrix for Y; it projects a vector onto the 2D space spanned by the columns of v1. It corresponds neatly to $latex H = u1\ u1^T\ $.)

$latex J = \left(\begin{array}{lll} 0.893533 & 0.26787 & -0.152898 \\ 0.26787 & 0.326045 & 0.384689 \\ -0.152898 & 0.384689 & 0.780423\end{array}\right)$

Let two test vectors y be :

$latex Y = \left(\begin{array}{lll} 2 & 2 & 2 \\ 0.5 & 1. & 1.5\end{array}\right)$

NOTE that my two vectors are the rows of Y, whereas my x vectors were the columns of X.

Instead of $latex \hat{X} = H\ X\ $, we have $latex \hat{Y} = Y\ J\ $ (because I put the vectors into the rows, and J is symmetric, J^T= J.) here is $latex \hat{Y}\ $:

$latex \hat{Y} = \left(\begin{array}{lll} 2.01701 & 1.95721 & 2.02443 \\ 0.485289 & 1.03701 & 1.47887\end{array}\right)$

and here is the difference between Y and $latex \hat{Y}\ $:

$latex Y - \hat{Y} = \left(\begin{array}{lll} -0.0170087 & 0.0427934 & -0.0244262 \\ 0.014711 & -0.0370126 & 0.0211265\end{array}\right)$

They are very close to the Y, of course. Again, you should assume that i searched high and low and tested hundreds or thousands of potential y vectors before selecting these two; and the computations dictated the selection, in contrast to my selecting first and then computing.)

Let's look at it the other way: what are the new components of Y? Of Yhat? To get new components, we multiply each vector $latex y_i$ by the inverse (i.e. transpose) transition matrix, i.e. by $latex v^T\ $. But matrix Y has vectors $latex y_i$ in rows, so we combine the individual

$latex v_i^T\ $,

which makes sense for column vectors $latex y_i$ into

Y v

which makes sense for row vectors making up Y, and this gives us the new components in rows. And just as we used u instead of u1 for this, we use v instead of v1.

$latex Y\ v = \left(\begin{array}{lll} -3.43684 & 0.430572 & -0.0521269 \\ -1.80694 & -0.482641 & 0.0450852\end{array}\right)$

The 3rd components are nearly zero, as they should be: the vectors in Y are almost in the 2D subspace spanned by v1.

As further reassurance, we may compute the new components of the $latex \hat{Y}\ $:

$latex \hat{Y}\ v = \left(\begin{array}{lll} -3.43684 & 0.430572 & 0 \\ -1.80694 & -0.482641 & 0\end{array}\right)$

Yes: their 3rd components are 0.

We need the "T" matrix for these Yhat. FIrst, give it a new name, S. (That's the second thing I wish Malinowski had done, because I take it to be a different matrix.)

We have the hat matrix J

$latex J = v1\ v1^T$

and the projections $latex \hat{Y} = Y\ J$

and instead of $latex \hat{X} = R\ T\ $, we will write

$latex \hat{Y} = S\ C = S\ v1^T$

so

$latex S = \hat{Y}\ v1\ $.

i.e. we compute

$latex S = \hat{Y}\ v1\ = \left(\begin{array}{ll} -3.43684 & 0.430572 \\ -1.80694 & -0.482641\end{array}\right)$

I have checked that $latex \hat{Y} = S\ C\ $.

This time we write

$latex D1 = R\ C = R\ S^{-1}\ S\ C = R\ S^{-1}\ \hat{Y}$

(The second reason for using S instead of T is that reverse, inserting $latex S^{-1}\ S$ instead of $latex T\ T^{-1}\ $.)

And now we use $latex \hat{R}$ to denote $latex R\ S^{-1}\ $, getting

$latex D1 = \hat{R}\ \hat{Y}\ $,

with

$latex \hat{R} = \left(\begin{array}{ll} 0.481412 & 2.03504 \\ 0.979059 & -2.02336 \\ 1.45929 & 2.08248 \\ 1.95929 & -1.91752 \\ 2.51976 & 1.89416\end{array}\right)$

Maybe i should point out that we inserted $latex T\ T^{-1}$ inside D1 = RC because $latex \hat{X} = R\ T\ $; we insert the reverse $latex S^{-1}\ S$ inside D1 = RC because $latex \hat{Y} = S\ C\ $.

approximate products

There is no doubt in my mind that Malinowski suggested replacing $latex \hat{X}$ by X in the equation

$latex D1 = \hat{X}\ \hat{C}\ $,

to get a different product which is approximately D or D1:

$latex D_X = X\ \hat{C}\ $.

I showed you that replacement in the previous post.

There is no doubt in my mind that it is equally valid to replace $latex \hat{Y}$ by Y in the equation

$latex D1 = \hat{R}\ \hat{Y}\ $,

to get a different product which is also approximately D or D1:

$latex D_Y = \hat{R}\ Y\ $.

Then, we could compare the original data matrix D with the two partially reconstructed data matrices $latex D_X$ and $latex D_Y\ $. Each of $latex D_X$ and $latex D_Y\ $ combines data (X and Y, respectively) with model ($latex \hat{C}$ and $latex \hat{R}\ $, respectively).

I have to say that malinowski does not describe the details of such a comparison between D and, say, $latex \hat{D}_X\ $.

The only doubt in my mind has nothing to do with my mathematics. What i did is valid, and plausible; but is it what he was recommending? I have to assume that his T in equation 3.133 in not the same T as in equation 3.136. To be specific, where he writes (3.132 and 3.133)

$latex X_{basic}\ \hat{Y}_{load} = \hat{D}\ \text{with } \hat{Y}_{load} = T^{-1}\ C$

I write instead

$latex X\ \hat{C} = \hat{D}_X \text{with } \hat{C} = T^{-1}\ C\ $.

All I did was change notation.

But where he writes (3.135 and 3.136)

$latex \hat{X}_{load}\ Y_{basic} = \hat{D}\ \text{with } \hat{X}_{load} = R\ T^{-1}\ \text{and with the same symbol } \hat{D}$

I write instead

$latex \hat{R}\ Y = \hat{D}_Y\ \text{with } \hat{R} = R\ S^{-1}\ $.

Not only is S not equal to T, but $latex \hat{D}_X \ne \hat{D}_Y\ $. They're close – that's the point of studying their deviations from D: they're both close to both D1 and D.

We could certainly have defined $latex \hat{X}_{load}= R\ T^{-1}\ $, but I can't see how we get anything useful from it. I have to believe that his T in the second pair of equations is a generic symbol, and that we really do have the two distinct computed matrices T and S.

Here are the two $latex \hat{D}$ matrices:

$latex \hat{D}_X = \left(\begin{array}{lll} 1.9793 & 3.04912 & 3.96366 \\ 1.01876 & -0.138675 & -0.952347 \\ 3.97801 & 4.98942 & 5.97121 \\ 3.01747 & 1.80162 & 1.0552 \\ 5.97671 & 6.92971 & 7.97875\end{array}\right)$

$latex \hat{D}_Y = \left(\begin{array}{lll} 1.98034 & 2.99786 & 4.01538 \\ 0.946439 & -0.0652406 & -1.07692 \\ 3.95983 & 5.00107 & 6.04231 \\ 2.95983 & 2.00107 & 1.04231 \\ 5.98661 & 6.93369 & 7.88077\end{array}\right)$

These are perfectly reasonable things to look at. X is close to $latex \hat{X}$ and Y is close to $latex \hat{Y}\ $; each of X, Y was chosen because it was close. Replacing $latex \hat{X}$ by X and $latex \hat{Y}\ $ by Y is a reasonable way to study the goodness of the model.

We would compute and investigate the differences $latex D - \hat{D}_X$ and $latex D - \hat{D}_Y\ $. Just to be clear, let me compute, instead, the difference between the two $latex \hat{D}$ matrices; they are not the same:

$latex \hat{D}_X - \hat{D}_Y = \left(\begin{array}{lll} -0.00104385 & 0.0512587 & -0.0517172 \\ 0.0723242 & -0.0734349 & 0.124573 \\ 0.0181767 & -0.0116541 & -0.0711045 \\ 0.0576412 & -0.199449 & 0.0128861 \\ -0.00989663 & -0.00397833 & 0.0979791\end{array}\right)$

sometimes approximate products

Finally, and this is what i think he meant by "the crown jewels of factor analysis" on p. 24, we could compute the product XY. We could also compute the product $latex \hat{X}\ \hat{Y}$ and the product $latex \hat{R}\ \hat{C}\ $. I remind you that when he writes $latex \hat{X}\ \hat{Y}$ on p. 64, he means what I've called $latex \hat{X}\ \hat{C}\ $. He has no symbol for the projection $latex \hat{Y}$ of his data $latex Y_{basic}\ $.

What do we know about those products? Well, XY should be close to $latex \hat{X}\ \hat{Y}\ $. But are they related to D or D1?

Well,

$latex \hat{X}\ \hat{Y} = R\ T\ S\ C$

so $latex \hat{X}\ \hat{Y}$ (and therefore XY) should be close to RC (hence close to D1 and D) if TS is the identity. By the same token,

$latex \hat{R}\ \hat{C} = R\ S^{-1}\ T^{-1}\ C = R\ {\left(TS\right)}^{-1}\ C$

so $latex \hat{R}\ \hat{C}\ $ should be close to RC if $latex {\left(TS\right)}^{-1}$ is close to the identity; that's equivalent to TS close to the identity.

It's time to write down all those products and relationships. We have the equalities

$latex D1 = R\ C = \hat{X}\ \hat{C} = \hat{R}\ \hat{Y}\ $;

and we have approximately

$latex \hat{X}\ \hat{C} \approx X\ \hat{C}$

$latex \hat{R}\ \hat{Y} \approx \hat{R}\ Y$

$latex D \approx D1$

so we could write

$latex D1 = R\ C = \hat{X}\ \hat{C} = \hat{R}\ \hat{Y}\ \approx X\ \hat{C} \approx \hat{R}\ Y\approx D\ $.

The other 3 products are

$latex X\ Y \approx \hat{X}\ \hat{Y}$

and

$latex \hat{R}\ \hat{C}\ $.

We have seen that all three of them will be close to D1 if TS is close to the identity. It's all or nothing for the three musketeers.

Well, let's just look at them. Here's XY...

$latex X\ Y = \left(\begin{array}{lll} 2.5 & 3. & 3.5 \\ 3.5 & 3. & 2.5 \\ 6.5 & 7. & 7.5 \\ 7.5 & 7. & 6.5 \\ 10.5 & 11. & 11.5\end{array}\right)$

and for comparison, we recall D...

$latex D = \left(\begin{array}{lll} 1.9 & 3.2 & 3.9 \\ 1 & -0.2 & -1 \\ 4 & 4.9 & 6.1 \\ 3 & 1.9 & 1.1 \\ 6 & 6.9 & 7.9\end{array}\right)$

They are not close. Just what is TS?

$latex T\ S = \left(\begin{array}{ll} 1.53043 & -0.128969 \\ -3.89709 & 0.99047\end{array}\right)$

Not close to the identity. $latex \hat{R}\ \hat{C}$ is no closer to D:

$latex \hat{R}\ \hat{C} = \left(\begin{array}{lll} 2.47532 & 4.69783 & 6.50672 \\ -1.00435 & -3.25667 & -5.00617 \\ 3.49907 & 5.74515 & 7.62871 \\ 0.0789657 & -2.08566 & -3.70896 \\ 4.37431 & 6.38245 & 8.13578\end{array}\right)$

Nor is XY:

$latex X\ Y= \left(\begin{array}{lll} 2.5 & 3. & 3.5 \\ 3.5 & 3. & 2.5 \\ 6.5 & 7. & 7.5 \\ 7.5 & 7. & 6.5 \\ 10.5 & 11. & 11.5\end{array}\right)$

Note that all of XY, $latex \hat{X}\ \hat{Y}\ $,$latex \hat{R}\ \hat{C}\ $ are quite different from each other, and quite different from D.

winding down

Please do not misunderstand me. $latex D_X = X\ \hat{C}\ $ and $latex D_Y = \hat{R}\ Y\ $ are always good things to look at. But XY, $latex \hat{X}\ \hat{Y}\ $, and $latex \hat{R}\ \hat{C}$ need not be.

Please do not misunderstand me: they could be. If we had just used a different Y matrix, namely

$latex Y0 = \left(\begin{array}{lll} 1 & 1 & 1 \\ 1 & 2 & 3\end{array}\right)$

instead of

$latex Y = \left(\begin{array}{lll} 2 & 2 & 2 \\ 0.5 & 1. & 1.5\end{array}\right)$

then everything would have worked out fine. We would have found that ST was very nearly the identity, and that $latex D \approx X\ Y$ (hence also $latex D \approx \hat{X}\ \hat{Y}\ $, and $latex D \approx \hat{R}\ \hat{C}\ $).

But the proof of the pudding is the product TS, where T comes from the X data and S comes from the Y data, and if they mesh, then they mesh. That is, if TS = I, then they mesh.

I have deliberately computed a case where they did not mesh. I deliberately messed up the Y matrix. I hope to make this failure more memorable than the possibility that they could have meshed.

The key is that any change of scale at all would have left me with a matrix Y which is still close to its projection $latex \hat{Y}\ $. If any vector y is close to the 2D subspace, so is any multiple of it. Such closeness by itself only guarantees that XY is close to $latex \hat{X}\ \hat{Y}\ $, but it says nothing about ST close to the identity, hence it says nothing about XY being close to D.

Now, if there were an underlying chemical or physical model such that D ~ XY, and if we chose among such X and Y data, then we might more than hope – we might expect – that things would mesh. The challenge is to get X and Y that go together. In fact, one would hope that simply choosing consistent units would make them go together.

Nevertheless, I'd rather you took a deep sigh of relief if they meshed rather than expect it.

Finally, I really did have X and Y that would have made everything work out peachy. Instead, I scaled the rows of Y, and although I could still get perfectly reasonable independent projections $latex \hat{X}$ and $latex \hat{Y}\ $, they were just that: independent.

As we wind down, I think you should asking: how did I know X and Y? Obviously they weren't real data.

For that matter, how did I construct D0, so precisely of rank 2? That's the question. And the answer.

I started with a 5x2 ("column") matrix

$latex \left(\begin{array}{ll} 1 & 1 \\ 2 & -1 \\ 3 & 1 \\ 4 & -1 \\ 5 & 1\end{array}\right)$

and a 2x3 ("row") matrix

$latex \left(\begin{array}{lll} 1 & 1 & 1 \\ 1 & 2 & 3\end{array}\right)$

and i multiplied them together to get D0.

$latex D0 = \left(\begin{array}{lll} 2 & 3 & 4 \\ 1 & 0 & -1 \\ 4 & 5 & 6 \\ 3 & 2 & 1 \\ 6 & 7 & 8\end{array}\right)$ = $latex \left(\begin{array}{ll} 1 & 1 \\ 2 & -1 \\ 3 & 1 \\ 4 & -1 \\ 5 & 1\end{array}\right)$ $latex \left(\begin{array}{lll} 1 & 1 & 1 \\ 1 & 2 & 3\end{array}\right)$

Then later on I took that column matrix for X and a scaling of that row matrix for Y.

Please do not think that we could have reconstructed X and Y. We invented them! In the real world, they are data to be searched out. We can talk about them here because this is a simulation. Remember example 6, and the fact that our X matrix was made up of independent data for o-xylene and p-xylene. In that case, we would need independent data for Y.

If they do not mesh, and if you cannot rescale things or find other data that does mesh, maybe you settle for studying

$latex D \approx \hat{D}_X = X\ \hat{C}$

and

$latex D \approx \hat{D}_Y = \hat{R}\ Y\ $

Of course, if you have only X data, then you study only $latex D \approx \hat{D}_X = X\ \hat{C}\ $, and if you have only Y data, then you study only $latex D \approx \hat{D}_Y = \hat{R}\ Y\ $.

Next will be a "summary" of Malinowski, but in fact I expect that it will be more in the nature of "compare and contrast". Although this example has introduced Y data and its projection $latex \hat{Y}\ $, I hope it was also a good review and summary of what we've done with Malinowski.

Questo livello di apprendimento del Taijiquan è quello riservato ai veri esperti e a chi, nel proprio cammino di apprendimento, ha saputo scoprire e fare propri tutti gli insegnamenti ricevuti e gli allenamenti seguiti.

A questo livello di conoscenza, lo studio del "cerchio" da piccolo diventa "invisibile", cosicchè anche la forma deve essere eseguita in maniera invisibile mostrando così la profonda interiorizzazione del movimento.

Per fare questo il corpo deve essere estremamente flessibile, così come l'adattabilità ad ogni minimo cambiamento deve essere immediata e totale. La "mutazione" delle forze Yin e Yang deve quindi avvenire internamente, così da risultare invisibile fuori. Il tutto si traduce in estrema velocità e flessibilità, doti fondamentali e necessarie ad affrontare una combattimento vittorioso.

Il Qi scorre liberamente, comunicando con il cielo e la terra allo stesso modo (concetto evidentemente proprio della filosofia taoista). L'artista marziale, a questo livello, in ogni momento della propria vita agisce in equilibrio mutevole con il Taiji (concetto sempre relativo alla filosofia taoista).

Vi è un perfetto equilibrio tra Yin e Yang. Il morbido e il duro si completano a vicenda, cosicchè i movimenti risultano sempre rilassati e vitali. Ogni parte del corpo è sensibile e può essere usata con la stessa potenza esplosiva (fa jing) di un pugno.

Il rapporto stabilito con il proprio corpo e la propria mente, cosi come tra i muscoli e gli organi interni, è di perfetto equilibrio.

Il praticante ha raggiunto così il bilanciamento perfetto tra Yin e Yang (50% e 50%).

Per le fonti originali di questo articolo vedi I CINQUE LIVELLI DI ABILITA' NEL TAIJIQUAN STILE CHEN di Gianna Sabatelli da "Arti d'Oriente" luglio-agosto 2002.