Sorry if this questions is overly simplistic. It's just something I haven't been able to figure out.

One of the most frequent occasions where linear systems of n equations in n unknowns arise is in least-squares optimization problems.
Let us look at an example. Let's say that we are studying two physical quantities y and x and we conjecture that y is a second order polynomial function of x, i.e. $y=\mathrm{\alpha <!--\; \alpha \; -->}{x}^2+\mathrm{\beta <!--\; \beta \; -->}x+\mathrm{\gamma <!--\; \gamma \; -->}$ for some real numbers $\mathrm{\alpha <!--\; \alpha \; -->}$, $\mathrm{\beta <!--\; \beta \; -->}$, $\mathrm{\gamma <!--\; \gamma \; -->}$ that are unknown.
Let's say now that we perform experiments and obtain measurements $(x_1,y_1)\cdots <!--\; \cdots \; -->(x_{100},y_{100})$. Applying the polynomial model on the measurements yields $y_i=\mathrm{\alpha <!--\; \alpha \; -->}{x}_i^2+\mathrm{\beta <!--\; \beta \; -->}{x}_i+\mathrm{\gamma <!--\; \gamma \; -->}$ for i=1,⋯100 or in matrix form Xk=y where $k=[\mathrm{\alpha <!--\; \alpha \; -->}\mathrm{\beta <!--\; \beta \; -->}\mathrm{\gamma <!--\; \gamma \; -->}{]}^T$, $y=[y_1\cdots <!--\; \cdots \; -->y_{100}{]}^T$ and the ith row of X is the row vector $[x_i^2{x}_{i}1]$. Now, as you might observe, we have 100 equations in 3 unknowns, i.e. our linear system Xk=y is overdetermined.
Practically speaking, this system is consistent (i.e. it has a solution) only if indeed y is related to x via a second order polynomial equation (i.e. our conjecture is true) and additionally there is no noise in our measurements.
So assume that none of the above two conditions is true. Hence the system Xk=y will not in general have a solution and one might consider finding a vector k that instead minimizes $||Xk-<!--\; -\; -->y|{|}_2^2$, i.e. the square of the error.
Then the solution of this optimization problem is the solution to the 3$\times <!--\; \times \; -->$3 system $X^{T}Xk=X^{T}y$. This formulation comes up all the time in engineering, e.g. in signal prediction. So, least squares problems lead to square (i.e. n$\times <!--\; \times \; -->$n) linear systems of equations.