New universal API with streaming/blocking endpoints (#990)

Previous title: Add api_streaming extension and update api-example-stream to use it

* Merge with latest main

* Add parameter capturing encoder_repetition_penalty

* Change some defaults, minor fixes

* Add --api, --public-api flags

* remove unneeded/broken comment from blocking API startup. The comment is already correctly emitted in try_start_cloudflared by calling the lambda we pass in.

* Update on_start message for blocking_api, it should say 'non-streaming' and not 'streaming'

* Update the API examples

* Change a comment

* Update README

* Remove the gradio API

* Remove unused import

* Minor change

* Remove unused import

---------

Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>

modules/extensions.py

@@ -14,7 +14,8 @@ def load_extensions():
     global state, setup_called
     for i, name in enumerate(shared.args.extensions):
         if name in available_extensions:
-            print(f'Loading the extension "{name}"... ', end='')
+            if name != 'api':
+                print(f'Loading the extension "{name}"... ', end='')
             try:
                 exec(f"import extensions.{name}.script")
                 extension = getattr(extensions, name).script
@@ -22,9 +23,11 @@ def load_extensions():
                     setup_called.add(extension)
                     extension.setup()
                 state[name] = [True, i]
-                print('Ok.')
+                if name != 'api':
+                    print('Ok.')
             except:
-                print('Fail.')
+                if name != 'api':
+                    print('Fail.')
                 traceback.print_exc()