Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[x/programs] Remove SmartPtr and HostPtr #872

Merged
merged 55 commits into from
May 2, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
25fba69
support program passing in param
iFrostizz Apr 25, 2024
ece7f6e
fix inter-program interaction
iFrostizz Apr 26, 2024
b01177f
update fixture
iFrostizz Apr 26, 2024
795ff58
keep using `SmartPtr` for now
iFrostizz Apr 26, 2024
a8a4f26
remove the need of a new Param type
iFrostizz Apr 29, 2024
f035e13
lint
iFrostizz Apr 29, 2024
2e1cdb4
remove param
iFrostizz Apr 29, 2024
942dbac
fix nits
iFrostizz Apr 29, 2024
71c4a34
remove whitespace
iFrostizz Apr 29, 2024
1c2d161
Merge branch 'main' into fix_program_calling
iFrostizz Apr 29, 2024
e7adafb
stop using SmartPtr
iFrostizz Apr 29, 2024
315d9df
fix tests
iFrostizz Apr 29, 2024
2194f8c
Cached state values in the program state (#840)
iFrostizz Apr 29, 2024
71ec384
stop using SmartPtr
iFrostizz Apr 29, 2024
21f1778
Cached state values in the program state (#840)
iFrostizz May 1, 2024
9d917bf
[x/programs] safe wrapper around C ffi interface (#869)
iFrostizz May 1, 2024
b6a426c
support program passing in param
iFrostizz May 1, 2024
11a9563
keep using `SmartPtr` for now
iFrostizz May 1, 2024
49532bf
remove the need of a new Param type
iFrostizz May 1, 2024
440022a
lint
iFrostizz Apr 29, 2024
c97bb64
stop using SmartPtr
iFrostizz May 1, 2024
2f0063d
fix tests
iFrostizz Apr 29, 2024
aebc12a
Merge branch 'main' into remove_pointer
iFrostizz May 1, 2024
285db8c
back off file
iFrostizz May 1, 2024
f7ef944
handle put and delete return as a status and remove the `SmartPointer`
iFrostizz May 1, 2024
3cbba41
cargo fmt
iFrostizz May 1, 2024
a26bb42
fix test
iFrostizz May 1, 2024
7e90e81
pass only context ptr as i32
iFrostizz May 1, 2024
cf8ebd4
fix token test
iFrostizz May 1, 2024
2255f8c
lint
iFrostizz May 1, 2024
a4ebe37
remove ignored lint
iFrostizz May 1, 2024
dc02edc
update fixtures
iFrostizz May 1, 2024
0174a25
move `CPointer` from `program` to `memory`
iFrostizz May 1, 2024
0ca123b
update fixutes to release
iFrostizz May 1, 2024
eac0dc8
cargo fmt
iFrostizz May 1, 2024
8ce44c9
Consolidate CI (#828)
richardpringle May 1, 2024
9dd246b
Update vm-release.yml (#876)
richardpringle May 1, 2024
3d76154
fix inter-program interaction
iFrostizz May 1, 2024
0f9494b
keep using `SmartPtr` for now
iFrostizz May 1, 2024
8f570b8
lint
iFrostizz May 1, 2024
e34eb7c
remove param
iFrostizz May 1, 2024
aff83f6
fix nits
iFrostizz Apr 29, 2024
4490172
stop using SmartPtr
iFrostizz May 1, 2024
82cbdf7
fix tests
iFrostizz Apr 29, 2024
cd30bf4
Cached state values in the program state (#840)
iFrostizz May 1, 2024
1d63869
stop using SmartPtr
iFrostizz Apr 29, 2024
49034b8
Cached state values in the program state (#840)
iFrostizz May 1, 2024
87a6416
[x/programs] safe wrapper around C ffi interface (#869)
iFrostizz May 1, 2024
89905d4
keep using `SmartPtr` for now
iFrostizz May 1, 2024
1d4d26e
fix tests
iFrostizz Apr 29, 2024
a12d9ae
handle put and delete return as a status and remove the `SmartPointer`
iFrostizz May 1, 2024
c5516bd
cargo fmt
iFrostizz May 1, 2024
e8764d3
move `CPointer` from `program` to `memory`
iFrostizz May 1, 2024
24bcb84
cargo fmt
iFrostizz May 1, 2024
d601a49
Merge branch 'main' into remove_pointer
iFrostizz May 1, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion x/programs/cmd/simulator/cmd/program.go
Original file line number Diff line number Diff line change
Expand Up @@ -144,7 +144,6 @@ func programExecuteFunc(
) (ids.ID, []int64, uint64, error) {
// simulate create program transaction
programTxID, err := generateRandomID()

richardpringle marked this conversation as resolved.
Show resolved Hide resolved
if err != nil {
return ids.Empty, nil, 0, err
}
Expand Down
17 changes: 10 additions & 7 deletions x/programs/examples/imports/program/program.go
Original file line number Diff line number Diff line change
Expand Up @@ -59,12 +59,15 @@ func (i *Import) Register(link *host.Link, callContext program.Context) error {
}

// callProgramFn makes a call to an entry function of a program in the context of another program's ID.
func (i *Import) callProgramFn(callContext program.Context) func(*wasmtime.Caller, int64, int64, int64, int64) int64 {
func (i *Import) callProgramFn(callContext program.Context) func(*wasmtime.Caller, int32, int32, int32, int32, int32, int32, int64) int64 {
return func(
wasmCaller *wasmtime.Caller,
programID int64,
function int64,
args int64,
programPtr int32,
programLen int32,
functionPtr int32,
functionLen int32,
argsPtr int32,
argsLen int32,
Comment on lines +65 to +70
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of casting all of these, I think you can just make them uint32 to begin with, can't you?

Don't worry about it for this PR, but see if you can avoid casting when tackle the rest of #862

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think that it's something that works in WASM, the only supported primitive types are i32, i64, f32, f64

Copy link
Contributor Author

@iFrostizz iFrostizz May 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm getting the following error: "function types incompatible: expected func of type (i32, i32, i32, i32, i32, i32, i64) -> (i64), found func of type (externref, externref, externref, externref, externref, externref, i64) -> (i64)"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's super weird. When using wasmtime directly in Rust, the only part of the type check that matters is the size. I just tried it out on the public_functions.rs code, and I swapped the allocate type signature to use u32 instead of i32, and it worked exactly as expected without changing the actual wasm.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, I just swapped the function parameters of the call_program linked function, I don't think that there is anything else to do ?

maxUnits int64,
) int64 {
ctx, cancel := context.WithCancel(context.Background())
Expand All @@ -80,15 +83,15 @@ func (i *Import) callProgramFn(callContext program.Context) func(*wasmtime.Calle
}

// get the entry function for invoke to call.
functionBytes, err := program.SmartPtr(function).Bytes(memory)
functionBytes, err := memory.Range(uint32(functionPtr), uint32(functionLen))
if err != nil {
i.log.Error("failed to read function name from memory",
zap.Error(err),
)
return -1
}

programIDBytes, err := program.SmartPtr(programID).Bytes(memory)
programIDBytes, err := memory.Range(uint32(programPtr), uint32(programLen))
if err != nil {
i.log.Error("failed to read id from memory",
zap.Error(err),
Expand Down Expand Up @@ -143,7 +146,7 @@ func (i *Import) callProgramFn(callContext program.Context) func(*wasmtime.Calle
}
}()

argsBytes, err := program.SmartPtr(args).Bytes(memory)
argsBytes, err := memory.Range(uint32(argsPtr), uint32(argsLen))
if err != nil {
i.log.Error("failed to read program args from memory",
zap.Error(err),
Expand Down
64 changes: 32 additions & 32 deletions x/programs/examples/imports/pstate/pstate.go
Original file line number Diff line number Diff line change
Expand Up @@ -43,38 +43,38 @@ func (*Import) Name() string {
func (i *Import) Register(link *host.Link, _ program.Context) error {
i.meter = link.Meter()
wrap := wrap.New(link)
if err := wrap.RegisterAnyParamFn(Name, "put", 3, i.putFnVariadic); err != nil {
if err := wrap.RegisterAnyParamFn(Name, "put", 6, i.putFnVariadic); err != nil {
return err
}
if err := wrap.RegisterAnyParamFn(Name, "get", 2, i.getFnVariadic); err != nil {
if err := wrap.RegisterAnyParamFn(Name, "get", 4, i.getFnVariadic); err != nil {
return err
}

return wrap.RegisterAnyParamFn(Name, "delete", 2, i.deleteFnVariadic)
return wrap.RegisterAnyParamFn(Name, "delete", 4, i.deleteFnVariadic)
}

func (i *Import) putFnVariadic(caller *program.Caller, args ...int64) (*types.Val, error) {
if len(args) != 3 {
return nil, errors.New("expected 3 arguments")
func (i *Import) putFnVariadic(caller *program.Caller, args ...int32) (*types.Val, error) {
if len(args) != 6 {
return nil, errors.New("expected 6 arguments")
}
return i.putFn(caller, args[0], args[1], args[2])
return i.putFn(caller, args[0], args[1], args[2], args[3], args[4], args[5])
}

func (i *Import) getFnVariadic(caller *program.Caller, args ...int64) (*types.Val, error) {
if len(args) != 2 {
return nil, errors.New("expected 2 arguments")
func (i *Import) getFnVariadic(caller *program.Caller, args ...int32) (*types.Val, error) {
if len(args) != 4 {
return nil, errors.New("expected 4 arguments")
}
return i.getFn(caller, args[0], args[1])
return i.getFn(caller, args[0], args[1], args[2], args[3])
}

func (i *Import) deleteFnVariadic(caller *program.Caller, args ...int64) (*types.Val, error) {
if len(args) != 2 {
return nil, errors.New("expected 2 arguments")
func (i *Import) deleteFnVariadic(caller *program.Caller, args ...int32) (*types.Val, error) {
if len(args) != 4 {
return nil, errors.New("expected 4 arguments")
}
return i.deleteFn(caller, args[0], args[1])
return i.deleteFn(caller, args[0], args[1], args[2], args[3])
}

func (i *Import) putFn(caller *program.Caller, id int64, key int64, value int64) (*types.Val, error) {
func (i *Import) putFn(caller *program.Caller, idPtr int32, idLen int32, keyPtr int32, keyLen int32, valuePtr int32, valueLen int32) (*types.Val, error) {
memory, err := caller.Memory()
if err != nil {
i.log.Error("failed to get memory from caller",
Expand All @@ -83,23 +83,23 @@ func (i *Import) putFn(caller *program.Caller, id int64, key int64, value int64)
return nil, err
}

programIDBytes, err := program.SmartPtr(id).Bytes(memory)
programIDBytes, err := memory.Range(uint32(idPtr), uint32(idLen))
if err != nil {
i.log.Error("failed to read program id from memory",
zap.Error(err),
)
return nil, err
}

keyBytes, err := program.SmartPtr(key).Bytes(memory)
keyBytes, err := memory.Range(uint32(keyPtr), uint32(keyLen))
if err != nil {
i.log.Error("failed to read key from memory",
zap.Error(err),
)
return nil, err
}

valueBytes, err := program.SmartPtr(value).Bytes(memory)
valueBytes, err := memory.Range(uint32(valuePtr), uint32(valueLen))
if err != nil {
i.log.Error("failed to read value from memory",
zap.Error(err),
Expand All @@ -116,10 +116,10 @@ func (i *Import) putFn(caller *program.Caller, id int64, key int64, value int64)
return nil, err
}

return types.ValI64(0), nil
return types.ValI32(0), nil
}

func (i *Import) getFn(caller *program.Caller, id int64, key int64) (*types.Val, error) {
func (i *Import) getFn(caller *program.Caller, idPtr int32, idLen int32, keyPtr int32, keyLen int32) (*types.Val, error) {
memory, err := caller.Memory()
if err != nil {
i.log.Error("failed to get memory from caller",
Expand All @@ -128,15 +128,15 @@ func (i *Import) getFn(caller *program.Caller, id int64, key int64) (*types.Val,
return nil, err
}

programIDBytes, err := program.SmartPtr(id).Bytes(memory)
programIDBytes, err := memory.Range(uint32(idPtr), uint32(idLen))
if err != nil {
i.log.Error("failed to read program id from memory",
zap.Error(err),
)
return nil, err
}

keyBytes, err := program.SmartPtr(key).Bytes(memory)
keyBytes, err := memory.Range(uint32(keyPtr), uint32(keyLen))
if err != nil {
i.log.Error("failed to read key from memory",
zap.Error(err),
Expand All @@ -148,7 +148,7 @@ func (i *Import) getFn(caller *program.Caller, id int64, key int64) (*types.Val,
if err != nil {
if errors.Is(err, database.ErrNotFound) {
// TODO: return a more descriptive error
return types.ValI64(-1), nil
return types.ValI32(-1), nil
}
i.log.Error("failed to get value from storage",
zap.Error(err),
Expand All @@ -163,7 +163,7 @@ func (i *Import) getFn(caller *program.Caller, id int64, key int64) (*types.Val,
return nil, err
}

ptr, err := program.WriteBytes(memory, val)
valPtr, err := program.WriteBytes(memory, val)
if err != nil {
{
i.log.Error("failed to write to memory",
Expand All @@ -172,18 +172,18 @@ func (i *Import) getFn(caller *program.Caller, id int64, key int64) (*types.Val,
}
return nil, err
}
argPtr, err := program.NewSmartPtr(ptr, len(val))
_, err = memory.Range(valPtr, uint32(len(val)))
if err != nil {
i.log.Error("failed to convert ptr to argument",
zap.Error(err),
)
return nil, err
}

return types.ValI64(int64(argPtr)), nil
return types.ValI32(int32(valPtr)), nil
}

func (i *Import) deleteFn(caller *program.Caller, id int64, key int64) (*types.Val, error) {
func (i *Import) deleteFn(caller *program.Caller, idPtr int32, idLen int32, keyPtr int32, keyLen int32) (*types.Val, error) {
memory, err := caller.Memory()
if err != nil {
i.log.Error("failed to get memory from caller",
Expand All @@ -192,15 +192,15 @@ func (i *Import) deleteFn(caller *program.Caller, id int64, key int64) (*types.V
return nil, err
}

programIDBytes, err := program.SmartPtr(id).Bytes(memory)
programIDBytes, err := memory.Range(uint32(idPtr), uint32(idLen))
if err != nil {
i.log.Error("failed to read program id from memory",
zap.Error(err),
)
return nil, err
}

keyBytes, err := program.SmartPtr(key).Bytes(memory)
keyBytes, err := memory.Range(uint32(keyPtr), uint32(keyLen))
if err != nil {
i.log.Error("failed to read key from memory",
zap.Error(err),
Expand All @@ -211,7 +211,7 @@ func (i *Import) deleteFn(caller *program.Caller, id int64, key int64) (*types.V
k := storage.ProgramPrefixKey(programIDBytes, keyBytes)
if err := i.mu.Remove(context.Background(), k); err != nil {
i.log.Error("failed to remove from storage", zap.Error(err))
return types.ValI64(-1), nil
return types.ValI32(-1), nil
}
return types.ValI64(0), nil
return types.ValI32(0), nil
}
14 changes: 7 additions & 7 deletions x/programs/examples/imports/wrap/wrap.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,22 +25,22 @@ type Wrap struct {
link *host.Link
}

// RegisterOneParamInt64Fn is a helper method for registering a function with one int64 parameter.
// RegisterAnyParamFn is a helper method for registering a function with one int64 parameter.
func (w *Wrap) RegisterAnyParamFn(name, module string, paramCount int, fn AnyParamFn) error {
return w.link.RegisterImportWrapFn(name, module, paramCount, NewImportFn[AnyParamFn](fn))
}

// AnyParamFn is a generic type that satisfies AnyParamFnType
type AnyParamFn func(*program.Caller, ...int64) (*types.Val, error)
type AnyParamFn func(*program.Caller, ...int32) (*types.Val, error)

// ImportFn is a generic type that satisfies ImportFnType
type ImportFn[F AnyParamFn] struct {
fn F
}

// Invoke calls the underlying function with the given arguments. Currently only
// supports int64 arguments and return values.
func (i ImportFn[F]) Invoke(c *program.Caller, args ...int64) (*types.Val, error) {
// supports int32 arguments and return values.
func (i ImportFn[F]) Invoke(c *program.Caller, args ...int32) (*types.Val, error) {
switch fn := any(i.fn).(type) {
case AnyParamFn:
return fn.Call(c, args...)
Expand All @@ -49,16 +49,16 @@ func (i ImportFn[F]) Invoke(c *program.Caller, args ...int64) (*types.Val, error
}
}

func (fn AnyParamFn) Call(c *program.Caller, args ...int64) (*types.Val, error) {
func (fn AnyParamFn) Call(c *program.Caller, args ...int32) (*types.Val, error) {
return fn(c, args...)
}

func NewImportFn[F AnyParamFn](src F) func(caller *program.Caller, wargs ...wasmtime.Val) (*types.Val, error) {
importFn := ImportFn[F]{fn: src}
fn := func(c *program.Caller, wargs ...wasmtime.Val) (*types.Val, error) {
args := make([]int64, 0, len(wargs))
args := make([]int32, 0, len(wargs))
for _, arg := range wargs {
args = append(args, arg.I64())
args = append(args, arg.I32())
}
return importFn.Invoke(c, args...)
}
Expand Down
2 changes: 1 addition & 1 deletion x/programs/examples/token.go
Original file line number Diff line number Diff line change
Expand Up @@ -386,7 +386,7 @@ func (t *Token) RunShort(ctx context.Context) error {
return nil
}

func (t *Token) GetUserBalanceFromState(ctx context.Context, programID ids.ID, userPublicKey ed25519.PublicKey) (res int64, err error) {
func (t *Token) GetUserBalanceFromState(ctx context.Context, programID ids.ID, userPublicKey ed25519.PublicKey) (res uint32, err error) {
key := storage.ProgramPrefixKey(programID[:], append([]byte{uint8(Balance)}, userPublicKey[:]...))
b, err := t.db.GetValue(ctx, key)
if err != nil {
Expand Down
6 changes: 3 additions & 3 deletions x/programs/examples/token_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ func TestTokenProgram(t *testing.T) {
t.Run("BurnUserTokens", func(t *testing.T) {
wasmBytes := tests.ReadFixture(t, "../tests/fixture/token.wasm")
require := require.New(t)
maxUnits := uint64(80000)
maxUnits := uint64(200000)
eng := engine.New(engine.NewConfig())
program := newTokenProgram(maxUnits, eng, runtime.NewConfig(), wasmBytes)
require.NoError(program.Run(context.Background()))
Expand Down Expand Up @@ -76,7 +76,7 @@ func TestTokenProgram(t *testing.T) {
// read alice balance from state db
aliceBalance, err := program.GetUserBalanceFromState(ctx, programID, alicePublicKey)
require.NoError(err)
require.Equal(int64(1000), aliceBalance)
require.Equal(uint32(1000), aliceBalance)

alicePtr, err = writeToMem(alicePublicKey, mem)
require.NoError(err)
Expand All @@ -92,7 +92,7 @@ func TestTokenProgram(t *testing.T) {

wasmBytes := tests.ReadFixture(t, "../tests/fixture/token.wasm")
require := require.New(t)
maxUnits := uint64(80000)
maxUnits := uint64(200000)
eng := engine.New(engine.NewConfig())
program := newTokenProgram(maxUnits, eng, runtime.NewConfig(), wasmBytes)
require.NoError(program.Run(context.Background()))
Expand Down
4 changes: 2 additions & 2 deletions x/programs/host/link.go
Original file line number Diff line number Diff line change
Expand Up @@ -136,12 +136,12 @@ func (l *Link) RegisterImportWrapFn(module, name string, paramCount int, f func(
// TODO: support other types?
valType := make([]*wasmtime.ValType, paramCount)
for i := 0; i < paramCount; i++ {
valType[i] = wasmtime.NewValType(wasmtime.KindI64)
valType[i] = wasmtime.NewValType(wasmtime.KindI32)
}

funcType := wasmtime.NewFuncType(
valType,
[]*wasmtime.ValType{wasmtime.NewValType(wasmtime.KindI64)},
[]*wasmtime.ValType{wasmtime.NewValType(wasmtime.KindI32)},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, this is why you can't use uint32... wasmtime-go doesn't support it. Casting should be fine in that case

)

return l.wasmLink.FuncNew(module, name, funcType, fn)
Expand Down
2 changes: 1 addition & 1 deletion x/programs/program/function.go
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ func (f *Func) Call(context Context, params ...uint32) ([]int64, error) {
return nil, err
}

result, err := f.inner.Call(f.inst.GetStore(), append([]interface{}{int64(contextPtr)}, callParams...)...)
result, err := f.inner.Call(f.inst.GetStore(), append([]interface{}{int32(contextPtr)}, callParams...)...)
if err != nil {
return nil, HandleTrapError(err)
}
Expand Down
Loading
Loading