to fix things. I know that TPTB gets "notes" from ABC. Maybe his vision and what their notes said--resulted in an impasse and they agreed to part ways.
I remember that happened with the creator and headwriter of "Revenge." He and ABC did not see eye to eye and so he bounced.
According to Deadline, Kelley had been unsuccessfully pushing the network and ABC Studios for shorter seasons, more like a cable network 13-episode model as opposed to the typical 22 episodes a season, which is creatively extremely tough to pull off -- as was evident in this uneven season of the dram--in order to ease pressure on cast and crew.