Adding emulator accounts from the cli

[10thfloor]

Issue To Be Solved

There should be a command to add an additional emulator account using the cli.

(Optional): Suggest A Solution

flow accounts add <optional name> should add an emulator account (not a testnet account as it is currently implemented) flow accounts add-balance <0xAddress> <amount> add FLOW to the specified emulator account.

(Optional): Context

In addition, new accounts could be populated in the flow.json file using the name specified. This would make it possible to sign transactions using the newly created account from the CLI immediately.

We might also want to consider moving this functionality out of flow.json altogether.

[bluesign]

Oh this would be great addition, I was just about the open an issue.

Context: I am trying to make dev-wallet to work with flow.json config, additional to FCL Wallet. Having ability to add accounts to flow.json is the missing part currently.

But there is a problem here with the emulator here: I think also there should be an option for emulator to auto create accounts from flow.json.

[devbugging]

I think also there should be an option for emulator to auto create accounts from flow.json.

That will be handled yes. After accounts are added to flow.json and emulator restarts by using CLI, the CLI will try to bootstrap emulator with flow.json state, so in case of accounts if there are more accounts defined in there it will try to create them, but that will only work if the accounts were created by the CLI in the first place, so no manual account creations as the addresses will not be sequential. It will be best effort bootstrap which will work in majority of cases but if the user is doing things manually then we can't bootstrap that, then I believe they should use state management and save the initial state. 😄 80% of the times it will work everytime

[bjartek]

I think the CLI should have an option to deploy all contracts specified on the emulator deployment block and create all accounts that are mentiond there that does not have contracts with them. I do this in GWTF with the following code

f.State here is from flowkit, saAccountName can probably just be removed and use "emulator-account"

// CreateAccountsE ensures that all accounts present in the deployment block for the given network is present
func (f *GoWithTheFlow) CreateAccountsE(saAccountName string) (*GoWithTheFlow, error) {
	p := f.State
	signerAccount, err := p.Accounts().ByName(saAccountName)
	if err != nil {
		return nil, err
	}

	accounts := p.AccountNamesForNetwork(f.Network)
	sort.Strings(accounts)

	f.Logger.Info(fmt.Sprintf("%v\n", accounts))

	for _, accountName := range accounts {
		f.Logger.Info(fmt.Sprintf("Ensuring account with name '%s' is present", accountName))

		// this error can never happen here, there is a test for it.
		account, _ := p.Accounts().ByName(accountName)

		if _, err := f.Services.Accounts.Get(account.Address()); err == nil {
			f.Logger.Info("Account is present")
			continue
		}

		a, err := f.Services.Accounts.Create(
			signerAccount,
			[]crypto.PublicKey{account.Key().ToConfig().PrivateKey.PublicKey()},
			[]int{1000},
			account.Key().SigAlgo(),
			account.Key().HashAlgo(),
			[]string{})
		if err != nil {
			return nil, err
		}
		f.Logger.Info("Account created " + a.Address.String())
	}
	return f, nil
}

// InitializeContracts installs all contracts in the deployment block for the configured network
func (f *GoWithTheFlow) InitializeContracts() *GoWithTheFlow {
	f.Logger.Info("Deploying contracts")
	if _, err := f.Services.Project.Deploy(f.Network, false); err != nil {
		log.Fatal(err)
	}

	return f
}

[devbugging]

@bjartek @10thfloor I believe the easiest way to solve a lot of problems related to this issue is to have an option to start the emulator with pre-created accounts, the CLI could have a start emulator flag for that and it would create X accounts on the startup and add it to the flow.json this accounts would be first X accounts on the network named "alice", "bob" etc. Would that solve problems mentioned? As long as creating manually added accounts go, I think that is another problem with another solution, @bluesign started some work there and as soon as I get some extra time I want to help on that, because that solution could also enable emulating testnet accounts on emulator.

[bjartek]

I have no issues with the flow you suggested but would really prefer if the new accounts added could be added to the deployments block with no contracts. That way overflow/go-with-the-flow will still continue to work without changes.

so from my perspective if this is an empty flow.json file

	"emulators": {
		"default": {
			"port": 3569,
			"serviceAccount": "emulator-account"
		}
	},
	"contracts": {},
	"networks": {
		"emulator": "127.0.0.1:3569",
		"mainnet": "access.mainnet.nodes.onflow.org:9000",
		"testnet": "access.devnet.nodes.onflow.org:9000"
	},
	"accounts": {
		"emulator-account": {
			"address": "f8d6e0586b0a20c7",
			"key": "5e9aaf23d1143642b7d9f2da8dd4a03930e1e5995581e04187e20bdbc6a4f22d"
		}
	},
	"deployments": {}
}

if you then start the emulator with --create-accounts 2 it should be like this

	"emulators": {
		"default": {
			"port": 3569,
			"serviceAccount": "emulator-account"
		}
	},
	"contracts": {},
	"networks": {
		"emulator": "127.0.0.1:3569",
		"mainnet": "access.mainnet.nodes.onflow.org:9000",
		"testnet": "access.devnet.nodes.onflow.org:9000"
	},
	"accounts": {
		"emulator-account": {
			"address": "f8d6e0586b0a20c7",
			"key": "5e9aaf23d1143642b7d9f2da8dd4a03930e1e5995581e04187e20bdbc6a4f22d"
		},
		"alice": {
			"address": "179b6b1cb6755e31",
			"key": "573b0db3fe91458e2aceefb8318d6daf3aee2af986a850cbf27a8ffff8a4ef9f"
		},
		"bob": {
			"address": "f3fcd2c1a78f5eee",
			"key": "573b0db3fe91458e2aceefb8318d6daf3aee2af986a850cbf27a8ffff8a4ef9f"
		}
	},
	"deployments": {
		"emulator": {
			"emulator-account" : {},
			"alice" : [],
			"bob" : []
		}
	}
}

So the flow json is very explicit about what it deploys/do. This is the way that overflow curently does things with the following code:

https://github.com/bjartek/overflow/blob/main/overflow/account.go#L12

[devbugging]

@bjartek I see. How come you have the need to have empty values in overflow? To be honest, I don't like setting default values as it fills flow.json with things that don't provide information, but I also don't like breaking your stuff, so if there are more good reasons for that I can consider this. Also, this is far from the final proposal. It's also worth asking what happens if such accounts are already filled in the flow.json but with some non-default deploy stuff, should this action be smart enough to at the same time deploy contracts there. Also another weird issue I see with your requirement is what happens then with other networks, how come you don't need default empty value for testnet, canary, mainnet?

[bjartek]

I do not like the solution that is there now either, but it is the only place in flow.json where you can signal that this account belongs to this network. If the accounts block could have that information then I would gladly remove it from the deployments block.

If I want to deploy accounts to testnet/mainnet for testing you could add them with empty values for contracts as well, but since accounts on testnet/mainnet are often blocto accounts then I do not use them that much.

Overflow will deploy contracts to all accounts that are populated with values in the deployments block.

[devbugging]

Ohh I see, so your problem is that you have no way of knowing whether an account should be/is created on testnet/mainenet/emulator? Do you think you could solve that by checking the address for validity based on the chain ID? https://github.com/onflow/flow-go-sdk/blob/master/address.go#L242 Never used this function before but I believe it should work for you to determine that.

[bjartek]

I use the AccountNamesForNetwork function on State (https://github.com/bjartek/overflow/blob/c7dc9d65f04b468c5cffeb64b147b62a96e07256/overflow/account.go#L19) in order to fetch the accounts and it uses the Deployment blocks to fetch it. So if that is changed it will just work I think.

[devbugging]

I understand. This is actually a mistake in the API, it should be getDeploymentAccountsForNetwork. So there are couple of options here I guess, one would be to add additional field to account network which is non-empty when account is non-emulator, another solutions is the one you are proposing, the third would be to figure out accounts network based on address. I believe since address encapsulates that information it might be best doing so from that, but, then the above mentioned function will change behavior and I need to check if that will be ok, anyhow like I said that function should be named differently based on what it really does.

[bjartek]

Sounds like a plan to me.

[bluesign]

Here we can maybe add one layer of "network" on accounts like other sections, if it is missing, we can fix flow.json on first run. ( detecting with chainID )

We can have 2 networks with same chainID in some cases maybe.

If this is good, I can make a PR for this ( with auto create accounts for network option, with emulator support )

I think passing Accounts to Emulator is the best option.

[devbugging]

@bluesign how do you mean a layer? do you mean an account in flow.json should have a property called network? what would that include and why do you think it's needed?

[bluesign]

@sideninja I meant something like below: ( instead of network property on account )

"accounts": {
            "emulator":{
		"emulator-account": {
			"address": "f8d6e0586b0a20c7",
			"key": "5e9aaf23d1143642b7d9f2da8dd4a03930e1e5995581e04187e20bdbc6a4f22d"
		}
           },
           "mainnet":{
             ....
           }
},

So we will have a cleaner grouping of accounts basically.

Also this allows even having multiple projects in one flow.json. You can define testnet-projectA , accounts and deployments for: testnet-projectA

PS: they don't have to be network names actually, I was more thinking in favour of a way to group accounts, and network made the most sense ( as it will be the most common usage )

if I run flow emulator --initial-accounts emulator it should create the missing ones for example.

[devbugging]

I see what you mean. The thing that I'm not too sure about is what the group will represent, because if it will represent a project then I would say you would typically create a separate flow.json for a different project like you create your own go.mod or package.json. If it was network information, I would say we could infer that from the address, couldn't we? So anytime we add another level to the flow.json I would like to have a good common argument, not a special case. Just because it might make the structure more complex. What do you think?

How do you see that creation working by the way? I have an idea lately that might be nice for local dev, but it's a bit different, I'm wondering how you see that working yourself.

[bluesign]

We can infer from the address, but it would be one more step to handle to keep it up.

I have something like this in my flow.json:

	"networks": {
		"canary": "access.canary.nodes.onflow.org:9000",
		"emulator": "localhost:3569",
		"mainnet": "access.mainnet.nodes.onflow.org:9000",
		"pentest": "dnz.dev:9000",
		"pentest2": "35.208.86.18:9000",
		"testnet": "access.testnet.nodes.onflow.org:9000",
          }

Main problem here is there for networks we use string as label. For example if I create an account in pentest, I have no idea if it should be testnet/benchnet/canary chain ID. If it is testnet chainID, I have no way of separating accounts with networks.

My use case is a bit extreme for sure. But I was thinking maybe we can have advanced mode for network like:

"pentest2": {
     Host: "35.208.86.18:9000",
     ChainID: "flow-testnet"
}

For emulator accounts parts:

I think some option to pass to emulator via Config ( https://github.com/onflow/flow-emulator/blob/c3d473dcedc724a48eb8313d272d052dfd5586e5/blockchain.go#L130 ) ( something like initialAccounts []Account ) and then when bootstrapping creating them would be the fastest.

[devbugging]

Yep adding the chain ID to the network using the advanced schema is valid.

How do you see emulators creating those accounts? especially if the addresses don't reflect the order of the used address generator?

[bluesign]

How do you see emulators creating those accounts? especially if the addresses don't reflect the order of the used address generator?

I was thinking putting Accounts to map[UInt64]Account (where key = account Index in generator) https://github.com/onflow/flow-go/blob/fc4eda0c5b1384bb27b98be43464f805552498a6/model/flow/chain.go#L136 Getting maxKey while putting. Then create accounts till currentIndex<=maxKey, adding defaultServiceAccount public key if Account doesn't exist in map.

[devbugging]

But you will need to be able to set them on chain as well wouldn't you?

Btw for development purposes, I have another idea, you usually don't care what the account address is as long as you can reference it, so I was thinking of allowing just specifying a name for the account and then (after import FLIP implementation) referencing those accounts by name. So you could have alice in flow.json without an address, it would be automatically generated, and then just reference alice anywhere you need it. Just an idea for now.

[bluesign]

But you will need to be able to set them on chain as well wouldn't you?

Yeah for example if you have in flow.json 4,5,10 account index accounts, I will create all accounts till index 10.

Btw for development purposes, I have another idea, you usually don't care what the account address is as long as you can reference it, so I was thinking of allowing just specifying a name for the account

Oh this is very nice, but a bit tricky. Imports etc will be ok, but Cadence parsing can be little tricky, for transactions etc:

getAccount(ALICE) should preprocess Alice there, no ?

[devbugging]

I see what you mean. So the LS will resolve the imports in order for Cadence to normally parse the code. In case when there will be getAccount directly in code, that I feel we could replace, although you are right this can be a bit tricky.

Yeah for example if you have in flow.json 4,5,10 account index accounts, I will create all accounts till index 10.

Ohh, I see yeah. hmm, I was thinking of this approach before, not sure what were concerns, will have to think about it.

[bluesign]

We can always create on first access, but it kinda feels like a 'hack'.

[devbugging]

Flow dev supper command is now handling creation of accounts and deployment, so closing this.