Ok, had a play with this and notice you can only add fields to new sections, not existing sections.
We really wanted to add additional address fields or category fields in the top section of a task.
Like you I’d want to order the fields based on the flow, might need to add one in between two in future as it would make more sense.