Interactive demo | kapi/2-complete

Intro

Generates text from predefined prompts using `Llama 2`.

Example::



    # returns string completion

    "What is Python?" | kapi.complete()

    # returns list of string completions

    ["What is Python?", "What is C++?"] | kapi.complete().all()

    # returns list of string completions. The prompts sent to the server are ["\n\n\nPlease summarize the above paragraph", ...]

    ["", ""] | kapi.complete("Please summarize the above paragraph").all()



- VRAM: 22GB

- Throughput: 8/s


Run
Server offline
Idle

Parameters

Result

(none yet)

Sample api request

Source code

        def inner(*args, **kwargs):                                              # batchify
            output = [...]; idx, event = con.add([output, args, kwargs])         # batchify
            t = threading.Thread(target=lambda: event.wait()); t.start(); t.join() # batchify
            return output[0]                                                     # batchify