Abstract: We explore the classical setting for the U(N) framework for SU(2) intertwiners for loop quantum gravity and describe the corresponding phase space in terms of spinors with the appropriate constraints. We show how its quantization leads back to the standard Hilbert space of intertwiner states defined as holomorphic functionals. We then explain how to glue these intertwiners states in order to construct spin network states as wavefunctions on the spinor phase space. In particular, we translate the usual loop gravity holonomy observables to our classical framework. Finally, we propose how to derive our phase space structure from an action principle which induces non-trivial dynamics for the spin network states. We conclude by applying explicitly our framework to states living on the simple 2-vertex graph and discuss the properties of the resulting Hamiltonian.