AI assisted remote work for heavy machines
Safety is one of the main requirements when carrying out various works. Machines are not yet able to work autonomously without the participation of people, but what prevents them from being controlled from a safe distance? Transition to remote control of a heavy machine, its productivity by 10-30%
Team: Case Eaters
Team membersIgnateva Tatiana, Robert Hidri, Ilya Molodchy
Members roles and background
Team leader : Robert Hidri. Robert is studying in Savonia University of Applied Science in Kuopio Finland(second year student, degree IOT(Internet of Things)). He’s 26 years old. Nationality: Albanian.Hobbies: reading and programming
Team worker: Tatiana Ignateva. Tatiana is studying in Saint-Petersburg State Forestry Technical University (fourth year student, degree information systems and technologies). She’s 21 years old. Nationality: Russian. Hobbies: frontend development and swimming
Acolyte: Ilya Molodchy. Ilya is studying in Saint-Petersburg State Forestry Technical University (fourth year student, degree information systems and technologies). He’s 21 years old. Nationality: Russian. Hobbies: programming and data analytics
Our company for this challenge was Mevea whose Digital Twins are mainly used:
1. To speed up development
2. For training human operators
We needed to find new possibilities of using DT?
One of the Mevea's services is to train in the virtual environment the future operators of heavy machinery.
What we did was add two points to the usage of Digital Twins by Mevea.
2.Ai testing environment
We wanted to use the capabilities that Mevea already had to facilitate remote work for heavy machinery operators.
Mevea would make the infrastructure where companies that use heavy machinery can do their work remotely .
The goal was to decrease the contact of workers with each other since we are in pandemic times .
But it would also lower the contact of workers with dangerous situations.
In order for this remote work to be possible and for the signal lag not to be a problem it would be assisted by AI. The AI system would fill in the role of the operator for the seconds that the signal lags from giving a command remotely to the machine doing the work. The operator would have the main control of the machine and the AI will fill in when needed.
The difference of our system from traditional remote operating would be using the training "simulator” + VR as the remote. The operators would feel like they are operating the machine and would get real time feedback from how the machine reacted to the environment and their commands.
Also with data that it gets by assisting the operator the AI would also train many times in "operating" the digital twin of the machines in the virtual environment.
The main benefits to this idea is to the safety of the workers. The danger from those kinds of works would decrease to near 0. Also the employers of the operators would not have to deal with the consequences that would come from accidents in the work environment. Also, there would be an increase in speed of the work because the safety measurements would not be so strict.
The use cases for this idea would be for machinery used in mining, dangerous terrain, and space.
The highest challenge would be high speed internet connection.
The success of the solution can be measured by the number of subscribers to the Remote Work capability that the challenge owner will offer to their customers.
The challenge owner will have a new unexplored marked. Until now there is no company that offers the same concept that we are thinking for the challenge owner.
The challenge owner already has the infrastructure in place to implement the solution, it will just need to add another use to that infrastructure.
The challenge owner will offer the infrastructure to machinery users against a fee like "zoom" for machinery workers. It will also have access to a lot of data from a bigger pool of machinery users and not just data from the producers and the training. This data can be used to advance autonomous system which would increase what theycould offer to their clients.
This hackathon/outside education unit based challenges can serve as a bridge between companies and universities.There is a divergence in what the students think what companies want and what the companies think the students should have. These educational challenges can make that divide smaller.This challenge helped us break the gap between us and companies.
During this experience we learned to work online using a digital ecosystem that we were not that familiar with.
We learned how to take a problem , divide it into smaller problems ,divide work between team members and prepare one comprehensive solution.
The teachers can measure the success of this education by the number of students that gain internships in different companies. We have to remind universities that the reason people go to university is to be able to find a job in real companies and not grades.
This hackathon and challenge like digital education can provide for teachers knowledge about where the industry is oriented to so they can teach their students the related knowledge. It can teach them how to make education closer to the work life. Make the connection work practical problems and school exercises.
The companies can gain a different point of view for the problems they want to solve. But it would also make the companies reassess their expectation of the student capabilities .And based on the good and bad points of the students that they find in this challenge they can make changes to the internship system in their company for better integration of students in their company .
Students can experience a real-life work situation in these challenges. They can find what companies are looking for and create a more realistic model of what to expect from the companies and work life in general.Students can learn what their knowledge is used for and what are the key areas where they can improve.The thing that this hackathon does best is it gives students like us contacts with companies where we can start looking for internships.
The success of the solution can be measured by the number connections formed between companies and students.
We have to find a new area where digital twin can find use.The solution was to make Mevea's digital twin and training capabilities in the platform of remote work for heavy machinery.
During this challenge the biggest problem was that we were not familiar with the same programs to work online and one of us had to adapt and learn the app that the majority used. This problems was solved by trying to us as much as possible the programs recommended by organizations.
Solution target group
Our soultion targets the users of heavy machinery used in different fields.
From mining to infrastructure and construction ,exploration,rescue missions.
Solution tweet textRemote work is here to stay. Why not make a better remote work experience even for heavy machinery operators. It can save their lives too.
Our solution is different in the fact that every one is looking at self driving capabilities but we are looking at what to do until we have self driving capabilities.There are remote control heavy machines but they are not user friendly and not efficient. The operator does not have the same control as he would have in our system.
We can give companies a part of the profit that they would have on self driving technology. We know that the future is autonomous systems but we also know that we arenot there yet and we need time to achieve full automation. Why not use this method to help in the advancement of AI(self driving) but also save human lives .
There are but most of the cases we find it in two pieces . Full immersion in controlling game machines in VR or controlling drones with a simple remote control and watching with VR headset the video feed from the drone camera.
The solution can be applied to all the situation that has to do with machinery .It can help the operators of those machinery stay at a safe distance but also get the full experience and efficiency as he would be driving himself.
The ability to remotely operate a machine from a safe environment while getting all the sensory data that you would get if you were on the machine opens a hugepossibility in the field of exploration, rescuing missions, mining industry etc. By removing the danger factor from different works, the risks taken would be higher and the advancement in every field would also be faster.
The solution in midterm would be to offer remote work to companies that already use Mevea services to train their operators. Because since most of the infrastructureis in place with the training equipment that will be the remote control module. The digital twins of the heavy machinery of their client and the virtual space to trainthe AI already are being used.
In the long term this service would spread not only to heavy machinery that their customer use but also to other industries where it is dangerous and not productive tooperate a certain machine in a certain environment. One example would be space mining where the worker would operate the machines from the remote control moduleinside the space ship without the need to get outside in a hostile environment but with the full sensory data that he would have if he was driving on the machine itself.
Solution team work
During this experience there were some challenges.
Like working remotely, in an ecosystem that you do not know really well, with people you can not communicate face to face and different levels of English.
But beside all these challenges we were able to make it work and finish our challenge in time and with the a product we liked.
We think that each one of us was interested in the topic and so were motivated to give our best to bring our idea to light.
This hackathon has been a great experience in part to the good team members we found. So we think that if there is a possibility in the future we would work together as a team again.
* Climate-KIC publishes the proposed solutions developed during the DigiEduHack event solely for the purposes of facilitating public access to the information concerning ideas and shall not be liable regarding any intellectual property or other rights that might be claimed to pertain to the implementation or use any of the proposed solutions shared on its website neither does it represent that it has made any effort to identify any such rights. Climate-KIC cannot guarantee that the text of the proposed solution is an exact reproduction of the proposed solution. This database is general in character and where you want to use and develop a proposed solution further, this is permitted provided that you acknowledge the source and the team which worked on the solution by using the team’s name indicated on the website.