This is a common knock on Keras, and it is valid, although sometimes I think it’s overblown. The ease of use of Keras often outweighs its inflexibility.
That said, sometimes TensorFlow is what you need, so partly it’s about using the right tool for the job.