One recurrent question about matrix computation is: What is the max dimension for a matrix operation, for example the determinant, or inversion, ?

Well, the right answer is: it depends. Many factors, such as hardware configuration, algorithm, software coding, operating system and - of course - the matrix, contribute to limit the max dimension . One sure thing is that the limit is not fixed at all.

In the past, the main limitation was memory and elaboration speed, but nowadays there is no more a limit.

We can say that, for the standard PC, the main limitation regards the 32-bit arithmetic and to the matrix itself.

Suppose you have a matrix (n x n) with its elements a_ij randomly distributed from -k to k

With this hypothesis the determinant grows roughly as:

where Log is decimal logarithm, n is the dimension of the matrix, k its max value

In 32 bit double precision the max value allowed is about 1E+300, 1E-300. So if we want to avoid the overflow/underflow error, we must take the constrain:

Plotting this relation for all points (k, n) we have the area for computing (blue area in the graph below); the dangerous error area is the remaining (white) area

How does it work?

Simple. If you have to compute the determinant of a matrix (80 x 80 ) having values no more than 1000, the point (1000, 80) falls into blue area; so you will be able to performs this operation

On the contrary, If you have a matrix (80 x 80 ) having values up to 1E+7, the point (1E+7, 80) falls into white area; so you will probably get an overflow error

From this graph we see that matrices (25 x 25) or less, can be elaborated for all values, while matrices (100 x 100) or more can be computed only if their values are less that 1000

Of course this result is valid only for generic matrices no ill-conditioned. For this special case you can get an overflow/underflow error even for low/moderate values. Fortunately, there are also special kind of matrices that can be elaborated even if the constrain (1) is false. We speak about diagonal, tridiagonal, sparse, block matrices etc.

But to get a result does not mean a "good result". We have to take care, specially for large matrices, to the round-off errors. They are quite underhand and very difficult to detect also; very often same results of large matrix inversion are take good even if they are wrong.

If you think that this errors regard only large matrices, have a look to the following example:

Compute the numeric inverse of this simple (3 x 3) matrix

If you use a in 32-bit standard precision program on your PC, the answer looks like the following:

And the determinant? You probably get a results near to DET = -6.867E-10

If you repeat the calculus with other programs you get similar results. Is there any reasons for suspecting this results? Yes, there is, because this result is completely wrong !.

In this case it was easy to analyze the matrix, but for a larger matrix (50 x 50) do you know what would happen?

Before to accept any results - specially for large matrices -we have to do same extra test.

In the example above we have to examine the SVD decomposition.

We have the diagonal matrix:

The last element is very near to the machine accuracy 1E-15, if we get the ratio between the lowest and the highest value we have:

The ratio is more less than machine accuracy , so we have to conclude that the matrix D, "numerically speaking" has one zero on the diagonal meaning that the given matrix is singular

Aug. 2002

Leonardo Volpi