A Machine Learning model takes tensors as inputs and outputs. Often you will want to submit and receive data in other formats, such as text or images. A transformer can be used to encode/decode the data as part of the Inference Service. Additional Transformer examples can be found in the KFServe Documentation.

The transformer in this repository encodes text for the GPT-2 model. You can use it without building by referencing pre-build images in Docker Hub.


The transformation logic is implemnted in preprocess is called on data before it is sent to model and postprocess on the result. In the GPT-2 example, the model is saved with signature name predict. This is added via the transformer for correct querying.

class Transformer(kfserving.KFModel):
    def __init__(self, name: str, predictor_host: str):
        self.predictor_host = predictor_host
        self.encoder = get_encoder()

    def preprocess(self, inputs: Dict) -> Dict:
        print (inputs)
        return {'signature_name':'predict','instances': [self.encoder.encode(instance) for instance in inputs['instances']]}

    def postprocess(self, inputs: List) -> List:
        return {'predictions': [self.encoder.decode(prediction) for prediction in inputs['predictions']]}


When making modifications you can easily build and upload your own version of the image to Docker Hub.

docker build -t dockerHubAccountName/imagename:0.1 .
docker push dockerHubAccountName/imagename:0.1

Last updated