Quantum signal processing and quantum singular value transformation are powerful tools to implement polynomial transformations of block-encoded matrices on quantum computers, and has achieved asymptotically optimal complexity in many prominent quantum algorithms. We propose a framework of quantum signal processing and quantum singular value transformation on $U(N)$, which realizes multiple polynomials simultaneously from a block-encoded input, as a generalization of those on $U(2)$ in the original frameworks. We also perform a comprehensive analysis on achievable polynomials and give a recursive algorithm to construct the quantum circuit that gives the desired polynomial transformation. As two example applications, we propose a framework to realize bi-variate polynomial functions, and study the quantum amplitude estimation algorithm with asymptotically optimal query complexity.