Unable to print to console without a newline

[yozachar]

I've a simple code which repeatedly gets data from https://httpbun.org.

(Slight modified version from: https://scrapecrow.com/asynchronous-web-scraping.html)

from httpx import Client, Response, ConnectTimeout
from time import time


def scrape(url: str, session: Client):
    """Scrape web for URLs."""
    return session.get(url)


def run():
    """Run program."""
    _start = time()
    results: list[Response] = []
    with Client() as session:
        # this url will always take 1 second
        url = "https://httpbun.org/delay/1"
        print("Sending request no.") # if I put `, end=""` it won't print.
        for idx in range(100):
            print(f"{idx+1} ", end="")
            results.append(scrape(url, session=session))
    print(f"\nFinished scraping in: {time() - _start:.1f} seconds")


if __name__ == "__main__":
    try:
        run()
    except ConnectTimeout:
        print("Oops! Timed out.")

But if I print, with end="", it only gets printed to the console when the program ends, why?

[tomchristie]

This isn't anything to do with httpx, but anyways...

Probably because of console buffering. If you don't want the newline perhaps you need to explicitly use flush=True?...

https://stackoverflow.com/questions/230751/how-can-i-flush-the-output-of-the-print-function

[yozachar]

Thanks! Wow, learning something new everyday.

https://stackoverflow.com/questions/25897335/why-doesnt-print-output-show-up-immediately-in-the-terminal-when-there-is-no-ne

[Charlotte-br560]

Hey there! Looks like you're running into a newline issue. To print without a newline, try using sys.stdout.write() instead of print(). And if you're looking for a robust solution, consider checking out Crawlbase for streamlined web scraping.