If a customer buys a larger quantity of items and that much quantity is not in stock then error message will be displayed in checkout.
The out-of-stock error message will only appear if there is at least one item available. If there is no stock available for an item, your customers will not be able to add it to their shopping cart so they will not proceed to the checkout flow.