The Englishmen, who had perhaps become accustomed to Lord Matsura's generous hospitality and his Japanese feasts, now began to complain about poor quality Chinese food and cheap drink in their own factory. For one man, called Jasper, the frustration boiled over and he hurled abuse at the Chinese housekeeper Li. When Li told Saris, the martinet captain ordered a boatman to punish Jasper with a whipping. However, the boatman neglected to carry out the order because he was much the worse for cheap drink. So the captain flogged both men himself. Perhaps Saris regretted his excessive severity. A few days later, he sent the men wines and cider from his private store, an inappropriate gesture in the circumstances.

On 2 July 1613, Richard Cocks visited the Dutch East India factory to try again to negotiate a fixed selling price for woollen cloth by the Dutch and the English. But Deputy Chief Factor Brouwer rejected his treaty. Now Saris was becoming frustrated and impatient for the arrival of William Adams. Saris had passed a letter for Adams to Lord Matsura's close retainer Sagawa Shuma and asked him to deliver it urgently. Sagawa Shuma had handed the letter to a messenger with instructions to deliver it to Adams, whether he was in Hemi or Edo. When the messenger left Hirado, Adams was in Hemi to celebrate the birth of his second child Susanna, or Kiku in Japanese. However, by the time the messenger arrived at Hemi, Anjin's family informed him that Ieyasu had summoned their lord to Sumpu. Curiously, the messenger did not leave Saris’ letter at Anjin's estate, nor did he call in at Sumpu. Instead, he went to Edo to report the Clove 's arrival to Shogun Hidetada and then returned to Hirado. It was Anjin's family who sent Adams an urgent message about an English ship's arrival at Hirado. Adams was annoyed that the messenger had not come to Sumpu on the way back to Hirado. Japanese sources suggest the messenger was instructed by Lord Matsura not to go to Sumpu, even though it was where Adams was most likely to be. On 12 July, Adams hurried out of Sumpu and headed for Hirado.