Smart light-responsive supramolecular materials have been extensively investigated in the past decade, but so far the impact of metal coordination on hierarchical supramolecular structures of lightresponsive building blocks has remained nearly unexplored. Herein, we unravel the hierarchical selfassembly of a small p-conjugated azo-containing pyridyl ligand that is able to respond to UV-light and metal complexation. The ligand self-assembles in an antiparallel fashion into long twisted fibers, which are then disassembled upon photoisomerization of the azobenzene groups, resulting in shorter rigid rods with a different packing motif. Complexation of Pd(II) ions enhances the cooperativity of the aggregation and induces a molecular rearrangement into slipped stacks with subsequent formation of long thin fibers. These are then transformed into thinner, shorter rods upon light irradiation. The observed different light-responsiveness, besides clearing up the influence of metal coordination and light irradiation in self-assembly processes, paves the way towards the design of novel supramolecular photochromic systems.