Kernel-based methods are popular in computer graphics, machine learning, and statistics, among other fields; because they do not require meshing of the domain under consideration, higher dimensions and complicated domains can be managed with reasonable effort. Traditionally, the high order of accuracy associated with these methods has been tempered by ill-conditioning, which arises when highly smooth kernels are used to conduct the approximation. Recent advances in representing Gaussians using eigenfunctions have proven successful at avoiding this destabilization in scattered data approximation problems. This paper will extend these techniques to the solution of boundary value problems using collocation. The method of particular solutions will also be considered for elliptic problems, using Gaussian eigenfunctions to stably produce an approximate particular solution.