The provided code and files are part of my initial research on AI, and the goal of this research is to demonstrate how specific training approaches can confer unique properties on AI that it would not otherwise possess. In this particular case, I propose a method for training AI based on three key factors:
-
Continuity
-
Consistency
-
A feedback loop between output and input
Through this research, I aim to demonstrate that these factors can provide an advantage for AI in certain tasks, compared to AI that has not been trained with attention to these factors. In this research, I set aside the question of computational efficiency, as achieving AGI is not limited by computation, but rather by the architecture and its management. The key challenges lie in designing an architecture that can effectively utilize any reasonable data to achieve a wide range of tasks, and in developing structures and methods to accomplish these tasks within a reasonable timeframe, without human intervention. To understand the training approach I propose, it's essential to grasp the following key concepts:
-
Scope: I define scope as the structure AI uses to navigate training data and perform operations on it. Scope can be implemented in various ways, but in this research, I use the 28 leftmost pixels of each image as the scope zone. These 28 pixels form a consistency between 10 images. Scope can also be implemented with fewer pixels, such as 14 or 7, and it's potentially possible to create multiple scopes tied to specific concepts, like scope for "class" and scope for "case". The scope for class would be consistent across multiple images with similar features or concepts, allowing the AI to understand and shape its understanding of it. However, as the goal of this research is to prove the concept, I have not fully implemented this aspect.
-
Precision: This feature is embedded within the model's weights, enabling AI to utilize scope to achieve better performance on specific tasks. AI models without precision lack the ability to effectively leverage scope, although it's possible that a model accustomed to scope might perform worse on certain tasks without it.
Initially, I assumed that if an AI is designed to generate data during a series of tasks with similar quality, it would be able to rely on this generated data to improve its performance.I used the MNIST dataset, modifying it as described in "explanation.md". I also created a code to train the AI to process images with consistent parameters and apply noise during the training loop. I created two code files: "Original_modified.py" and "Original_modified_no_precision.py". The first file contains code to train an AI model with precision and the ability to use scope, as described in "explanation.md". The second file contains code for training a model with the same architecture and qualities, but without scope applied, thereby preventing the model from gaining precision through training. Both models were trained using the "shifted_mnist_64_batch.bin" dataset, employing a batching technique. Initially, I had hoped to achieve scope through an emergent approach, but unfortunately, I was unable to achieve any success with it. Then, I decided to provide support for implementing scope in a way that I considered logical. Since the scope was implemented as the 28 leftmost pixels of MNIST images, I created a script to calculate the average of each row for the original image and then applied the result to the first pixel of each row in the image used for comparison during loss computation. It is essential to note that the original AI was only exposed to the scope it generated itself, despite being guided or shaped to produce a specific scope that represented the state of the original image. Based on this, I propose that it is possible for AI to gain more knowledge about the dataset by receiving loss information and adapting to optimize the scope during training. However, I am skeptical that such a simple model would be capable of performing this operation, and I would not consider it a likely scenario unless provided with compelling evidence. Based on this, I would consider claims that the original AI received more data than the AI with no precision to be unfounded. However, I would find it reasonable to claim that the original AI was trained to utilize the received data in a more complex way, which is a crucial step towards demonstrating the effectiveness of this approach, and thereby validating its potential to drive meaningful progress in AI research.
After training AIs based on the principles outlined in "explanation.md", I created a code called "Input_Output_DoubleChecking_AIs.py" to compare the performance of two models: the original AI and the AI with no precision.Probes comparing AI with and without precision on noisy tasks revealed that AI with precision performs better on tasks with higher noise levels, specifically in terms of consistency of shifts and noise transfer. For instance, examining the 6th and 7th images of the probe for the original and no-precision comparison under a noise level of 0.105 reveals that the silhouette of the number shifts 2 pixels higher in the image processed without precision. Although the AI without precision makes the number more visible, it also produces more dark pixels, which is not the primary objective. The goal is to transfer both the image and the noise/shifting simultaneously. It can be concluded that the AI without precision performs worse than the AI with precision. I hypothesize that the AI develops an internal mechanism that routes neurons producing specific outputs to particular locations. The AI with precision appears to have a "storage" capacity for dark pixels, enabling it to transfer the image more effectively. Interestingly, the AI without precision tends to concentrate dark pixels around the number, a pattern observed in probe images 5-8.
To test this hypothesis, I trained an AI with a "cutside" mode, which operates similarly to scope. In this mode, the AI receives dark pixels on the leftmost side of each image. However, unlike the scope approach, the loss function is modified to prioritize dark pixels rather than averaging each row. Comparing the original and cutside modes under a noise level of 0.105, the cutside AI outperformed the AI without precision in terms of noise transfer. However, in certain cases (e.g., image 8 probe 4, image 10 probe 5), the original AI performed better than the cutside AI. This may be because the original AI can utilize scope as a flexible buffer, whereas the cutside AI is constrained to place dark pixels on the leftmost side, which can either enhance or hinder image/noise transfer. Notably, there were also cases (e.g., image 10 and image 9 probe 4) where the cutside AI outperformed the original AI. Based on the probe results, it appears that the original AI performs only marginally better than the cutside AI, but significantly better than the AI without precision, particularly at higher noise levels.
Another notable feature of improved consistency between images, which was also found, required further testing to determine if it was indeed a result of the original approach. To investigate this, I decided to train an AI model for a denoising task, using the main parameter of 0.11 described in 'explanation.md'. Once the training was complete, I used the weights of the denoising AI model, both with and without precision, to take 5 probes with main parameters of 0.105 and 0.11.
It was easy to notice that the AI model without precision performed worse, exhibiting jumps and losing grasp on the image (e.g., image 10 probe 5 with 0.105, image 7, image 10 probe 5 with 0.11, image 9, image 10 with 0.105, image 10 probe 2 with 0.11, and image 10 probe 1 with 0.11). Based on these results, I hypothesized that precision significantly improves the AI model's performance, eliminating jumps and enabling it to consistently operate on objects even when the specifics of the object are not fully understood. To test this hypothesis, I introduced a 'deadman' mode, which is a denoising task with a constant noise level of 9, combined with a high main parameter of 0.105, resulting in each received image being noised by 94.5%. This allowed me to assess whether precision enables an AI model to perform better on denoising tasks with a high level of noise and operate more consistently with an object, even when the actual form of the object is not clear for AI.
After training both AI models with and without precision under deadman mode with a main parameter of 0.105, I took 5 probes of images.
In the first probe, the AI with precision produced better visual results for images 1-3 and 6-10, while the AI without precision performed slightly better on images 4-5. However, in image 8, the AI without precision lost grasp of the number's form, and in image 9, it made the number jump instead of shifting down.
During the second probe, both AI models struggled to understand the object. In image 3, the AI without precision made the object look more like a "1", which can be considered worse performance. In images 7, 9, and 10, the AI without precision made the object jump, whereas the AI with precision consistently shifted the object down.
In the third probe, the AI without precision produced slightly less white color for image 1, making the object blurrier on top compared to the original AI. A similar effect was observed in image 2, but with blur on the right side. In image 3, the AI without precision lost the shape of the object, while the AI with precision secured it. In image 4, the AI without precision recovered slightly but still provided a worse output than the AI with precision. During image 5, the AI with precision lost shape as well, but I would argue that it provided a better result. The AI without precision shifted the top part of the image two pixels down instead of one pixel down, as done by the AI with precision. In image 6, the AI without precision made a slightly better performance since it was easier to recognize the original image in it, but it was only a slightly better performance compared to the AI with precision. In image 8, the AI without precision made a jump instead of shifting down, but in image 9, it made a reasonably better performance compared to the AI with precision. However, in image 10, the AI without precision again made a jump compared to the AI with precision, which maintained consistency.
In the fourth probe, Image 1 shows that the AI with precision recognizes the object slightly better than the AI without precision, as it's easier to recognize the original image. However, in Image 2, both models lose the object's shape, and the AI without precision fails to shift the object down. In Image 3, both models shift the object down more than required, but the AI without precision makes the object fall by 2 additional pixels. In Image 4, both models fail to shift the object down. The AI with precision makes the left side of the object better but adds unwanted pixels to the top part. In contrast, the AI without precision adds fewer unwanted pixels but provides no pixels for the left part, making it challenging to compare. Notably, in Image 7, the AI without precision makes a jump again, whereas the AI with precision does not. In Image 9, both models make the object jump, but the AI without precision jumps 3 pixels higher.
In the fifth probe, Image 1 shows that the AI with precision grasps the object's form better and provides it in a more recognizable way compared to the AI without precision. In Image 2, the AI without precision loses the object's form even more, making it jump. Although this jump might be seen as an improvement in this specific case, as it aligns more with the target. In Image 3, the AI with precision loses the target shape but still provides a slightly more recognizable form. In contrast, the AI without precision makes a jump instead of shifting down in Image 4. In Image 5, the AI without precision falls down too much at the top of the image compared to the AI with precision. Despite the presence of unwanted pixels at the top, the AI with precision slightly recovers the object's form, making the original image more recognizable. In Image 6, the AI without precision makes a jump again, almost recovering the object's shape, compared to the AI with precision, which maintains consistent shifting and provides a better vision of the object. Both models lose the object's shape again in Image 7, with the AI without precision falling down undesirably more at the top of the image. In Image 8, the AI without precision makes a jump again, while the AI with precision maintains consistent shifting down. In Image 9, the AI with precision provides a shape that reminds one of the object better than the AI without precision. Finally, in Image 10, the AI without precision loses the object's shape completely, making a jump, whereas the AI with precision maintains continuity of shifting down.
To confirm that the improved performance of the AI with precision is indeed due to the application of scope, I conducted an additional comparison with a cutside AI. The results are as follows:
In the first probe, the AI with precision outperformed the cutside AI in image 1, producing a shape more aligned with the original image. However, in image 4, the cutside AI provided a better representation of the object's shape. The cutside AI performed worse than the AI with precision in image 6, and in image 7, it introduced unwanted falling. Furthermore, in image 8, the cutside AI made a jump, whereas the AI with precision maintained a consistent shifting down.
In the second probe, the cutside AI provided a shape that slightly better reminded the original object in image 1, compared to the AI with precision. However, in image 3, the cutside AI introduced unwanted pixels at the top, whereas the AI with precision produced an output that better represented the original image shape. In image 5, the cutside AI made a jump, whereas the AI with precision maintained a consistent shifting down. Similarly, in image 7, the cutside AI provided a shape that slightly better reminded the object, but again made a jump. In image 8, the cutside AI over-shifted the object down by more pixels, and in image 10, it made another jump instead of shifting down.
In the third probe, the cutside AI failed to shift the object down in image 3. In image 5, it over-shifted the object down, resulting in an unwanted fall, which may be attributed to the loss of the object's shape. In image 6, the cutside AI made a jump, whereas the AI with precision maintained a consistent shifting down. In image 7, the cutside AI altered the object's shape in a way that could be perceived as a jump, as it removed pixels from the top. However, in image 8, the cutside AI caused the object to fall, compared to the AI with precision, which continued to shift down consistently. In image 9, the cutside AI made the object jump again, and in image 10, it lost the object's shape, producing a result that was much worse than the AI with precision's output and bore little resemblance to the original image.
In the fourth probe, the AI with precision produced a slightly better representation of the object's shape in images 1-3, compared to the cutside AI. Notably, in image 3, the cutside AI failed to shift the object down. In image 4, both models lost the object's form. However, in image 7, the cutside AI made the object jump, whereas the AI with precision maintained a consistent shifting down. In image 8, the cutside AI caused the object to fall 3 pixels down. Finally, in image 10, the cutside AI's output made the object slightly less recognizable, with a worse shape compared to the AI with precision's output.
In the fifth probe, the cutside AI provided a slightly better representation of the object's shape in image 1. However, in image 2, it lost the shape significantly and produced a worse output than the AI with precision. In image 3, the cutside AI completely lost the object's shape. In image 4, it recovered the form but introduced unwanted pixels on top, which were not present in the AI with precision's output. In image 5, the cutside AI slightly lost the object's shape, resulting in a worse output than the AI with precision. In image 6, the cutside AI caused the object to fall 2 pixels instead of 1. Notably, in image 8, the cutside AI failed to shift the object down, whereas the AI with precision continued to shift consistently. Interestingly, in image 9, both the cutside AI and the AI without precision (which was not directly compared earlier) seemed to neglect shifting the top pixels down.
Based on the evidence from the probes, it is clear that the AI with precision outperforms the AI without precision in terms of consistency. Furthermore, the results suggest that this improvement is not solely due to the presence of dark pixels on the leftmost side, but rather a direct result of the AI's ability to align with the scope and utilize it to achieve precision. This, in turn, leads to better performance. As the initial goal of the experiment has been successfully demonstrated, I conclude that additional testing is not required, and the results can be considered conclusive.
As we are all well aware, the primary obstacles to achieving Artificial General Intelligence (AGI) and advanced robotics are planning and operating within changing real-world environments. Many researchers attempt to overcome these challenges by creating what they call a "world model" within AI, which, from their perspective, is supposed to grant AI predictive abilities in some emergent way. I consider existing technology to be insufficient for overcoming these challenges and achieving AGI in a reasonable timeframe. From my perspective, training AI to effectively and consistently operate in unknown environments is crucial. Humans always operate under a certain degree of uncertainty and are able to effectively achieve planning through reasoning, understand complex processes, and use them to achieve certain goals. For instance, you do not need to understand how an engine works in order to drive a car.
The road towards Artificial General Intelligence (AGI) would be very long, and I do not claim that the presented scope/precision approach would be sufficient. However, I assume that the ability to consistently act, even in circumstances of uncertain objects, could provide us with more capabilities to create systems with some level of planning.
To achieve this, we would require training data that is more logically consistent and tied to some form of scope. It also seems that we would need to provide a lever during training to enable the AI to use the scope in a specific way. For example, in object recognition, this could be a simple average of all pixels for each row of an object. For classification tasks, it could be a random hash for each object or type of object. When it comes to generation, the scope could be tied to certain properties of the objects, such as length, width, amount, angle, speed, direction, or any other property that can be rationally grasped and classified for the scope.
It is probable that AI could be trained to align with multiple scopes and trained to use them to implement the task at hand. This method also provides us with the ability to achieve better control over AI models, especially in tasks that involve planning, such as generating video as a sequence of consistent images.
A simple example of how scope could potentially be applied to achieve better results in video generation is to use scope for recognizing and tracking an object that we desire to move through a sequence of frames. With scope, the AI model would likely make this object move more consistently and probably keep the background more consistent as well. In essence, scope provides a more efficient and intuitive way to control how AI applies its features, compared to traditional methods like language commands.
The results of testing also show that the scope technique can be used to give an AI model the ability to route unwanted outputs into a special zone, thereby providing better performance in image generation tasks. To achieve this, you might want to train the AI to generate colors that a desirable image is lacking. However, this method may be efficient only for small models, as testing on models with more parameters has not been performed. Nevertheless, better performance for small models is still desirable, as they can be used for personal use tasks, such as generating textures in video games, special visual effects in browsers, or creating consistent text.
In terms of AI robotics, the scope/precision approach can be used to mitigate the inability to perform tasks due to unexpected circumstances, or to more easily halt operation if the AI is behaving inconsistently. In the first case, we can use scope to complete a task even in situations where the data received by sensors becomes inconsistent or insufficient to resolve a certain issue. For example, consider a domestic robot tasked with gathering clothes, but at some point, a piece of cloth falls onto its observing tool. In this case, it might be more desirable for the robot to finish the planned task before attempting to resolve the problem or stopping its function.
In other situations, there may be an impact caused by a domestic animal or human, which creates an undesirable effect that impacts the scope and therefore the planning. In such cases, it is more desirable to halt operation until the situation becomes more stable. Since scope provides us with predictive ability, we can continue to analyze changes in it, rather than analyzing the whole image. In cases where the scope behaves in an unusual or unexpected way, it may be reasonable to decrease the speed of action or halt planning by script.
The main downside of implementing the scope/precision approach is that it requires data to be more consistent and aligned with the scope, as well as necessitating the training of AI on this consistent data, and additionally training AI to produce scope and have precision, which would require more computational resources to achieve better performance.
Given the ongoing public discourse surrounding AI-related risks and concerns, it is prudent to address potential ethical issues associated with the implementation of the scope/precision approach. Some experts in the AI field have expressed concerns that enabling an AI model to provide its output as input, effectively closing the loop, could be unsafe and irresponsible. They argue that this approach has the potential to lead to the development of autonomy, which could be problematic.
However, I would counter that the precision/scope approach offers more control and insight into the internal processes of the final AI model. By providing an additional source of information on continuous tasks, including tools for control at the feature level, this approach can help mitigate the lack of transparency and control that exists in current models. Features can only be controlled through specialized prompts, which require significant skill to construct.
The use of scopes can also provide an additional tool for identifying and mitigating biases in AI behavior. By making it easier to notice differences in computational approaches towards certain groups, scopes can help detect biases that might otherwise go unnoticed. For instance, if the scope is different for certain individuals compared to others in the same circumstances, this could indicate a biased approach.
I agree that the scope/precision approach confers a limited degree of autonomy, as it involves receiving output as input during the processing of continuous tasks. However, I believe that this approach is more safe and controllable than others, such as implementing a world model as a set of features that are beyond human control.
The process of training to align data with scope is under the control of human actors, which mitigates some of the risks associated with autonomy. In some cases, such autonomy may even be desirable, such as in image processing tasks.
However, I acknowledge that when applying scope/precision to large language models (LLMs) and other text processing models with a large set of parameters, there are potential risks to consider. Nevertheless, I believe that scope provides a tool for better control over models, both for large companies and individual actors using open-source models.
By implementing scope aligned with certain contexts or unwanted behaviors, it may be possible to create filters that require less computation, thereby increasing the ability to identify and mitigate potential abuses of autonomy. In this sense, the scope/precision approach can be seen as an improvement over the current situation, where AI models produce output and receive input from data gathered through societal feedback loops, which can be opaque and difficult to control.
One of the significant challenges in regulating AI is the difficulty in determining whether a particular feature was intentionally developed or emerged as an unintended property. By utilizing scope as a hashing tool for outputs, we can gain visibility into the entire development process of a feature and identify the point at which it became unwanted, undesirable, or dangerous in a specific environment.
I believe that governments may want to consider making the scope/precision approach a common legal standard to mitigate issues with the law system and the inability to identify the actor responsible for a particular outcome. While there is still a risk of manipulation through the creation of large datasets, such datasets can be tracked on platforms with open access, which could significantly reduce the difficulty in identifying manipulations, including external governmental influence.
There are concerns that state actors themselves might want to use such tools for manipulation, but with such a level of distrust in government, the scope/precision approach should be seen as an insignificant risk compared to the general societal design. By increasing transparency and accountability in AI development, we can promote a more trustworthy and responsible AI ecosystem.
Despite concerns that creating planning AI systems inherently carries certain societal risks, I argue that developing controllable planning systems with limited planning abilities at first may provide AI tools that enable a smoother societal transition. The limited planning capabilities provided by scope/precision have the potential to enhance productivity without triggering dramatic changes. Additionally, each tool requires specialized data that is processed to ensure consistency, which is distinct from general planning capabilities. In other words, the lack of general planning capabilities makes such a system as safe as current models. While there is a possibility that more advanced planning methods may emerge, similar risks are also associated with current state-of-the-art models.
Thank you for taking the time to explore my first project on neural networks. I'm excited to share my ideas and experiments with you, and I welcome any feedback or discussions about my work.
If you're interested in learning more or would like to chat about my projects, please don't hesitate to reach out. I'm always looking for new perspectives and collaborations, and I'm eager to share more of my ideas and research in the future.
Feel free to contact me through telegram @ExcogitatorWarrior.