But the health secretary says he will not negotiate on pay, adding that the union has lost goodwill with their five-day walkout.