Secure Node Tracking and Payment System Software Development Project Discussion


#21

Yes that is the proposed scenario right now. The staking t_address could be in the same wallet for all the nodes, but it needs to be a separate t_address per node. It gets a bit more complicated if we tried to tie multiple nodes to the same t_address.

The caveat is that payments will need to be made to the staking address, otherwise someone could use an someone else’s address for the stake and collect zen on their node. Making payments to the staking address discourages this, but does not really enforce using your own address if you want the payments to go to someone else who is staking it.

There is a requirement of a minimal amount of zen on the node itself to pay any transaction fees a challenge incurs. Perhaps 1 or 2 zen which should cover it for a long time (at .0001/transaction).

The staking address is not associated with the secure node’s wallet.dat. It is part of the configuration of the pool app and used to identify the node and make payments from the payment system on the pool server.


#22

OK - but does the challenge transaction fee have to come from the same t_address as the registered node?


#23

The design calls for the fee to come from the first t_address of the node. The balance is checked before the challenge. If there are some security concerns we can look at a different design.


#24

This makes sense now. I spaced on the multi-address / wallet part.

I am thinking of creating an AWS AMI that will have the secure node and associated security settings preloaded to lower the entry barrier for those trying to set up a secure node.
One would load the image, configure t_address and secure node wallet, send a few ZEN for challenge transaction fee, and be good to go. Are there any efforts toward this already?


#25

Using a big cloud provider like AWS to host secure nodes for a distributed system sounds all kinds of wrong. I can understand why someone would want to do it to make money, but it will harm the project by making it trivial for the establishment to track traffic and have access to multiple nodes simultaneously.

We want geographically distributed nodes, run by competent people who keep them updated and secure. We do not want a click-and-go solution enabling people who don’t know what they are doing to weaken the project in their attempt to make money from it.


#26

I am new to cryptos, but extensive experience in business, dev and IT.

As far as I understand secure nodes and communication between them should be built in such way that it doesn’t matter where they are hosted or who is monitoring them. So it is not an issue whether they are hosted on AWS or any other cloud provider, giving they are deployed and configured properly. “Weak” nodes can be hosted in super secure data center, still they can be compromised.

If there is monetary incentive, of course there will be different people (also dishonest) who are willing to earn money by running secure nodes. In my opinion the dev team should come up with solution that nodes that are run by any type of person are secure and risk of compromising any node is minimal.


#27

Overall, I strongly disagree with the ‘Node Pool’ proposal. Some of the issues it tries to fix are not wrong in the first place and the overall approach seems insecure and contrary to the goal of having a resilient, distributed system. The idea that a single central location has an ‘always on’ connection to all secure nodes out there makes a mockery of the whole project - in my eyes anyway. What happens if the port where all nodes are connecting to is DDoS-ed?

I will apologise in advance if I am a bit too passionate in my responses below, and again apologies if I have misunderstood something and as a result criticised it. I can’t help it - this proposal reminds me of a poorly made sales pitch with little basis in facts.

In more detail:

Every time a node fails to provide a response to a challenge, it will be considered to be down for the period before the challenge. Simple, and accurate enough.

How to fairly calculate payments with up-time being one of the criteria

Allow one response per challenge from each node, pay per correct response.

How to test minimum requirements periodically

Checking that the 42ZEN are present and certs are correct is trivial. The challenge-response system takes care of much of the rest. If any specific parameter is problematic it would be good to know which one it is.

How to track the health of the network such as nodes that are out of sync with the blockchain

The only way I can think of is by exposing some RPC commands to the tracker - the question is do we expose them directly through an ‘always on’ connection or in a controlled and limited way through the SecNodeApp.

An always connected separate tracking system based on a pool model has many benefits:
No additional port needed on the node to published responses eliminating one attack vector. Sockets are established outbound from the node server.

This additional port would be opened to very few addresses only - one for each tracking server as we should have more than one. It only becomes an issue if the tracking server is compromised - an in that case it would be a MUCH smaller issue than if there was an ‘always on’ connection.

Completely independent pool network that does not affect the zen network

The case with the previous system as well - how is this a genuine benefit? This has a much greater potential effect on the zen network as all ‘secure’ nodes are always connected to a single central location.

Server does not have to constantly poll its own address on the blockchain for responses, transaction id of challenge could be sent to server from node and verified

I am not sure I understand - which server has to poll its own address constantly? The tracking server collects the responses from the nodes X seconds after the challenge is published. This is an intermediate solution until the ultimate goal of publishing the responses to e.g. IPFS or GNUNet as a way to check the secure publishing functionality is developed.

Reduced number of challenges and simplified challenges resulting in fewer administrative transactions in the block chain. The node is a known validated client.

It is not obvious to me from the text, but I assume the reduced number of challenges is a result of the always on connection being used to track uptime, so less granular challenges needed? If that is the case we are replacing a relatively way of tracking uptime with a much more complex disconnection tracking. How do you envisage that specifically, per-second uptime tracking with disconnects over 24 hours accumulated to a total number of seconds? Yes it would track uptime more accurately but it will be much more complex than counting valid responses to challenges.

Self registering nodes with minimal configuration

How does the pooled system reduce the configuration required on the nodes? If nodes are self-registering, how would the pool verify they are actual nodes and not fake ones or DOS agents?

Always connected nodes determine uptime for payments with a high degree of granularity and in a fair manner. Allowances for node maintenance could be factored in.

Yes, uptime tracking would be the most accurate in a centralised pool system, but do we need that level of accuracy? We are not building a VOIP conference system where every second counts. I would argue we do not need it, not at the cost of making the system centralised and insecure.

Availability of node metrics to monitor the health of the pool network and the zen network

Some level of exposure of RPC commands to the tracker may be unavoidable, but I would definitely want to expose a limited subset of these in a controlled way through the node app. Having access to RPC commands over an always on connection sounds like a recipe for disaster.

Alerts to node operators for items such as low balance, software upgrades, out of compliance, etc.

This would be good, yes, and can be implemented regardless of the way tracking is done so is not really a benefit of this system.

Two way communication between the node and server with automatic reconnect

Sounds scarily insecure, a bit like a windows network waiting to get owned.


#28

Ideally, yes. In reality that is a goal almost impossible to achieve. Can you give me an example of a single piece of (comparably complex) software which has a minimal risk of compromise when incompetently administered? Tor possibly, but the time, effort and money poured into it is not on par with the resources available to ZEN.

I am not saying they should create weak software and rely on node owners to secure it; equally though it is unreasonable to expect the devs to foresee every silly thing an admin somewhere can come up with. We should try to do the best we can on both sides.

I would suggest the IP ranges of AWS and similar very large cloud providers and VPS providers be tracked and a maximum of 20-30% of the nodes to be permitted to come from those ranges as a way to ensure better distribution of nodes and system resilience.


#29

I agree with you that the system should be distributed on not centralized.

But I also would like to see Secure Nodes up and running first so we can see what the interest level and economics of them are. This is a Minimum Viable Product implementation of tracking, reporting, and payment.

By getting the Node Pool version of the sytem running in a short period of time, we can then work to design, test, and deploy a more distributed and resilient version as an update.


#30

nikmit,

Your response is exactly the kind of response I was hoping to get. I want to see as many holes punched into any design as much as possible so it can be strengthened and modified as needed. I would be concerned if no one challenged the design. All ideas and concerned are welcomed. And I appreciate your passion. It is part of what makes a strong community.

It may get too confusing trying to interlace responses within your responses, so I’ll address the major ones, combine where appropriate and number the points so they can be referred to in responses.

1 DDoS server: this would be true of the polling design or pool design. Both are centralized systems. There would be only one URL for a Foundation server to either poll for challenges or to connect to the pool. In either design an assumption can be made that all known secure nodes where up during a server attack and should be paid (conditions need to be defined).

2 DDoS node: in the polling design having a web server sitting on each node for the sever to check for the challenge response creates a whole new attack point. There is no additional open port in the pool design since the node pool app initiates the connection to the server.

3 Uptime-availability: the polling design only describes the node looking for a challenge, with the node itself providing no identity and the server not tracking who is polling. Let’s say a challenge is the only way to identify a node and there is only one challenge a day. What is there to stop someone from checking for a challenge and only running the node during the challenge (not sure why they would unless the system is used for other purposes)? Or even writing an app to mimic a node. To combat that, the number of challenges would likely be increased. Another approach is to have the server track each node that polls the server by ip address and associate it with an identity to prove it is up, but that does not prove the node itself is running.

In the pool design, identity (t_address of node and a challenge during registration) is checked during initial connection. The server knows who the node is since the connection is maintained. Node information (like block height and number of peers) is checked periodically to ensure the node is running. Regarding the point of tracking uptime to the second: that is not really the point, although possible I guess. The system would track downtime (disconnects) and calculations would include allowances for things like maintenance. We don’t want to penalize nodes.

4 Uptime-tracking: I am not sure where the assumption came from on tracking disconnects each second in the pool design. The server would simply notice a disconnect event, create a record of start time and when it reconnects add a stop time to the record. There should be very few of these records. It could even allow a ‘grace period’ and ignore disconnects shorter than a few minutes to take into account things like network issues or reboots. I would anticipate very few disconnect records.

5 Node health: in a polling design, some information like block height or peer count could be added and transferred in the challenge. Or perhaps in the polling itself, but some identity would need to be added. The pool design keeps tabs on these items or anything that would help with the health of system.

6 RPC on the node: in both designs the tracking app is making calls to the node via rpc. The code is limited to reading information for node metrics and challenges and submitting a shielded transaction of 0 amount. The control is at the node level and there is nothing that can be run outside of what is hard-coded. Having an instruction to perform a challenge sent via a secure socket or pulled via https does not change how it is executed on the node.

7 Server polling own address: yes this is a little confusing.

In the polling design, the information (URL of the node to find the response it created) is in a shielded transaction memo sent to the server. After a challenge period ends, the server has to loop through each of the new transactions in its z_address to get the URL of each node. It then needs to query the node at that address, retrieve the response, validate it and log the results.

In the pool design when the node is done with the challenge it sends back the transaction id. The server looks up the transaction by id, validates it and logs the result. Additionally if the challenge can not be completed by the node, the error is sent back to the server, logged and available for display so it can be addressed.

8 Self registering: the amount of configuration would be similar in either design. The self registering simply means when the pool app running on the node starts up, it connects to the server, the server determines it has not seen it before, the server sends it a challenge to perform, the server validates and registers it. Data like block height, software versions, and peer count also are used to determine if it is a secure node. Some sort of SSL validation will be included once that is finalized.

9 Using the block chain for administration: Let’s assume there needs to be six challenges a day (one every four hours) to ensure node availability in the polling design. At 1000 nodes that is 6000 transactions a day. At 5000 nodes, 30,000 transactions a day. Going into the wallet on the server.

This could be more than standard transactions depending on number of nodes. It also just does not seem like good practice to flood the blockchain with lots of administration data. Some is needed, but these numbers are excessive. The pool design does not depend on the blockchain to determine uptime. It also could be set to only send a challenge every few days or when the node is down for a time since the challenge is only used to prove capability.

10 Scalability: The polling design requires times of peak activity. The pool design spreads out activity. In fact most of the time it is just sitting there monitoring the sockets and getting node metrics. Challenges can be spread out over the course of a day or even a week. Handling the number of target nodes could be done with one server at the outset.

11 Two way communication: To address the little bit of FUD on this one: The pool design could use a package like socket.io to provide and maintain the underlying encrypted socket connections. It has ‘heartbeat’ and timeout features built in. This package has been downloaded from npm over 5.7 million times in the last month indicating there are many applications using and relying on it in production settings. The pool design is not being built completely from scratch at that level, it would use well-known and supported libraries. And no, it is not like a windows network.

Overall I believe the pool design addresses many of the implementation issues/challenges I found while writing the detailed spec.

I hope I have addressed your questions and provided cleared explanations. Let me know if I missed anything. This has also been a good exercise for me in being challenged to rethink aspects of each design. Keep the questions, concern and ideas coming!


#31

I’m having trouble understanding where you’re coming from. “The establishment” is going to track traffic? Who is the establishment? Amazon? Please. At best they would see a bunch of cyphertext. And even that would likely violate dozens of SLAs.

AWS has data centers all over the world.

One could describe AWS’s entire business model is being updated, secure, and accessible. Are you expecting people to build a server in their garage and set up a secure node on there? That’s quixotic at best.

Your exclusivity is alarming. With your proposed level of qualification ZenCash can expect to reach a critical mass of secure nodes about the same time we finish colonizing Mars.

In my proposed scenario, there’s one gold standard VM image that is ready to run a secure node on a secure system out of the box. The same image can undergo scrutiny from the community and made to be a hardened gold standard. This image would foster decentralization and adoption by allowing more people to access it.

Getting security right is extremely difficult for billion dollar corporations with teams of highly qualified security professionals. 1000+ seperate enthusiasts getting security right is insanity.

I must be missing something here because the more I read into your response the more it seems completely backwards. Why would you want it intentionally difficult to set up and operate a secure node? How could we ever hope to reach a critical mass of secure nodes? There’s barely enough qualified people in the world to do what you’re suggesting, and the tiniest fraction of them has even heard of ZenCash. Please tell me I’m misunderstanding something here.


#32

nikmit, I total agree with you. But I think we are in a minority on this opinion. The Idea of a server being operated by someone with no experience is a scary thought to me.


#33

I was thinking, for nodes to sign up, they should publish a transaction with an address that has 42 zen on it and a unix timestamp. That way it’s decentralized and anyone can count how many nodes exist. We can use a new opcode like OP_NEWNODE which checks the address for having >= 42 zen and also check the unix time isnt in the future. We’ll also create a new opcode called OP_RENEWAL which does the same thing as OP_NEWNODE but requires the previous OP_NEWNODE/OP_RENEWAL transaction.

Example script:
zsfa9VVJCEdjfPbku4XrFcRR8kTDm2T64rz 1500091482 OP_NEWNODE which would create a transaction with 20170bab9f1db3c8fc73eb2d09daa6f872819e782ce74926fbc12bc5782c1bc9 as its txid

then

zsfa9VVJCEdjfPbku4XrFcRR8kTDm2T64rz 1500151482 20170bab9f1db3c8fc73eb2d09daa6f872819e782ce74926fbc12bc5782c1bc9 OP_RENEWAL to renew the node next month.

Downside to this is that it doesn’t actually verify securenodes. Anyone with 42 zen in one address can create these. It’s just for people scanning the blockchain to get a rough idea on how many people are running securenodes.


#34

Brandon, there are all levels of inexperience, I suspect there will be people that have never even used a command line interface, have no clue what a script is, and if they can’t use there mouse will be lost; and they will want to run a SN. Your idea will allow then to point and click their way to their own SN. The idea of that is disturbing to me.
Now I will switch sides for moment and say that I do have some experience with “risk analysis” (how much should we spend to keep it secure), and it seems that the risk if a SN is compromised is pretty low, especially if the 42 zen will not be kept on the SN. Since the SN code is freely available its just as easy to stand up your own node than try to compromise someone else node.
So while the idea of someone with no experience running a node is a disturbing idea to me, I don’t ultimately see much down side. (the one draw back to a whole bunch of image identical nodes is, if a vulnerability is found, someone could potentially bring down a huge number of nodes at once)


#35

JDP -
Thanks for your response.
I believe I originally came off harsher than I intended to - passion is passion. :slight_smile:

You and nikmit certainly have good points that in theory I agree with. It just seems extraordinarily impractical. In order for ZenCash to gain traction, it needs to be easier to contribute to the network.

Where my imagination is failing me is the alternative to lowering the barrier of entry.
How do we get a high number of secure nodes if there’s a high level of sophistication required?
And, if such sophistication is required, isn’t that a form of centralization?

Lastly, even with my proposed deployment strategy, there is still a level of technical prowess required. Using ssh and a shell, for example, would still be a requirement.


#36

I dislike cloud computing on a general basis as it centralises the internet and makes it so much more corruptible. In the words of Peter Sunde, “Big data is like big tobacco. Presumed to be safe for decades and when we understood the cancer it brought, it was hard for people to quit.”

I would not want to have cloud computing at the basis of our brand new freedom-loving establishment-resisting system.

The degree here is what makes or breaks things. As I said before, I’d be OK with AWS and other major cloud/VPS providers being used for nodes if the total number of nodes hosted there is not greater than 30% of all nodes. Do you think that makes sense, or do you think it will be safe even if all nodes are on AWS? Where do you draw the line?

I expect people to build and host nodes in whichever way they are most comfortable with - if the garage is someone’s thing then why not. Reductio ad absurdum doesn’t contribute to a constructive debate, and you have done that with my post on more than one occasion. That said I I don’t take it personally - a clash of opinions can be stimulating if the end goal is getting closer to truth rather than getting the upper hand.


#37

This is exactly why our official eleos wallet runs with its own full node instead of relying on others. To prevent network centralization


#38

I would bet every person who views this page is a fan of decentralization, and I certainly see your concern with hosting a majority of nodes on a single VPS provider.

I am absolutely open to alternatives, but I haven’t seen many presented.
There are 2 sticking points that I can’t think of an alternative for:

  • A better security solution than a hardened image. It would be resource efficient to pen test a single image rather than a network.
  • How to get 1000+ nodes in the near future without lowering barrier to entry. There are only 1600 people in Slack.

Cheers


#39

Given this, would it be a tragedy if most secure nodes were hosted on a cloud provider? As I think about it more and more I think it would it be fine.


#40

The problem with a secure hardened image is that nothing is impenetrable, and the wider the adoption of that image is the more it becomes a target. If it ever did get compromised the effect on the network will be much more severe as multiple or even a majority of nodes will be affected. So on a risk management chart we are moving from a high/medium likelihood and low impact to low/very low likelihood and high/very high impact. I don’t see that as an easy choice.

In terms of adoption we could run a poll in slack and see what people’s plans are for running secure nodes and get a feel for the problem we are trying to solve.