The answer is: it depends. There are some great non-L Canon lenses, and certainly some great third party lenses.
For example, the 85mm 1.8 Canon lens is a fantastic lens. But it's not an L. Another quality non-L is the 70-300 IS USM. It's very sharp. BUT, it's plastic. And it's not as fast to focus as some of it's L siblings.
The L designation will provide better construction quality - metal instead of plastic. In some cases it provides weather sealing. The L zooms tend to have better image quality performance throughout the zoom than non L lenses of the same era. THey also are more likely to maintain a wide aperture throughout the zoom than non-L Canon lenses. There are L primes which outshine any consumer lens - or third party lens. There's nothing out there that matches the 400mm 2.8 or 300mm 2.8 or 500mm f4 for image quality and focus performance. There is no lens on the market to my knowledge which matches Canon's 70-200 f4 IS for that type of lens. Fantastically sharp. Even the non-L is a wonderful lens with no consumer grade counterpart. Sharpnes,s focus speed and build quality are pro grade.
Having said all that - taking a landscape shot at 25mm with 18-55 IS kit lens at f11 vs. 16-35 f2.8 at the same settings and you'd be hard pressed to see any difference in the results.