Massachusetts sued five major banks Thursday over deceptive foreclosure practices such as the “robo-signing” of documents, potentially undermining negotiations between lenders and state prosecutors across the nation over the same issue.