it depends on your priorities as a person and a creative. if it's stability, go for in-house/agency, but keep in mind that progression is kinda slow in this field and there's little to no room for creativity (depending on clients/team you work with). in terms of growth as a designer, i've learned so much working in an agency setting compared to in-house. there's less hand holding too in agencies because everything is fast paced so there's that :/
i'm lucky that our creative director doesn't mind us doing freelancing on the side. so yeah graphic design has its pros and cons and you really need to have the drive, discipline, and the right network* (like any other profession i guess) to progress.