Singular Matrix

[LennyAharon]

since we are simply trying to add very small epsilon and our variance is very large we end up dealing with a singular matrix issue in some cases. The solution will be the following:

under stats.py add regulation term to make sure matrix is invertible

    B = np.zeros((v.shape[0], W.shape[1], W.shape[1]))
    n_latent_dim = W.shape[1]
    for i in range(v.shape[0]):
        # Add regularization term to ensure matrix is invertible
        matrix_to_invert = W.T @ np.diag(1.0 / (v[i] + epsilon)) @ W + epsilon * np.eye(n_latent_dim)
        B[i] = np.linalg.inv(matrix_to_invert)

Also add this fix to the last part

    M = {view_idx: np.zeros((v.shape[0], 1)) for view_idx in range(num_views)}
    for i in range(v.shape[0]):
        for view_idx in range(num_views):
            idxs = slice(2 * view_idx, 2 * (view_idx + 1))
            # Add regularization to ensure Q matrix is invertible
            Q_reg = Q[view_idx][i] + epsilon * np.eye(2)
            M[view_idx][i] = diff[i, idxs].T @ np.linalg.inv(Q_reg) @ diff[i, idxs]