Update Java Cloud Debugger to v2.2

Author: vladlf

README.md

@@ -1,16 +1,181 @@
 # Java Cloud Debugger
 
+Google [Cloud Debugger](https://cloud.google.com/tools/cloud-debugger/) for
+Java.
+
+## Overview
+
 The Cloud Debugger lets you inspect the state of a Java application at any code
 location without stopping or slowing it down. The debugger makes it easier to
 view the application state without adding logging statements.
 
 You can use the Cloud Debugger on both production and staging instances of your
 application. The debugger adds less than 10ms to the request latency only when
 the application state is captured. In most cases, this is not noticeable by
-users.
+users. The Cloud Debugger gives a read-only experience. Application variables
+can't be changed through the debugger.
+
+The Cloud Debugger attaches to all instances of the application. The call stack
+and the variables come from the first instance to take the snapshot.
+
+The Java Cloud Debugger is only supported on Linux at the moment. It was tested
+on Debian Linux, but it should work on other distributions as well.
+
+The Cloud Debugger consists of 3 primary components:
+
+1.  The debugger agent (requires Java 7 and above).
+2.  Cloud Debugger backend that stores the list of snapshots for each
+    application. You can explore the API using the
+    [APIs Explorer](https://developers.google.com/apis-explorer/#p/clouddebugger/v2/).
+3.  User interface for the debugger implemented using the Cloud Debugger API.
+    Currently the only option for Java is the
+    [Google Developers Console](https://console.developers.google.com). The
+    UI requires that the source code is submitted to
+    [Google Cloud Repo](https://cloud.google.com/tools/repo/cloud-repositories/).
+    More options (including browsing local source files) are coming soon.
+
+This document only focuses on the Java debugger agent. Please see the
+this [page](https://cloud.google.com/tools/cloud-debugger/debugging) for
+explanation how to debug an application with the Cloud Debugger.
+
+## Options for Getting Help
+
+1.  StackOverflow: http://stackoverflow.com/questions/tagged/google-cloud-debugger
+2.  Google Group: cdbg-feedback@google.com
+
+## Installation
+
+The easiest way to install the debugger agent is to download the pre-built
+package from the Internet:
+
+```shell
+mkdir /opt/cdbg
+wget -qO- https://storage.googleapis.com/cloud-debugger/compute-java/debian-wheezy/cdbg_java_agent_gce.tar.gz | tar xvz -C /opt/cdbg
+```
+
+Alternatively you can build the debugger agent from source code:
+
+```shell
+git clone https://github.com/GoogleCloudPlatform/cloud-debug-java.git
+cd cloud-debug-java
+chmod +x build.sh
+./build.sh
+```
+
+## Setup
+
+The Java Cloud Debugger agent is a
+[JVMTI](http://docs.oracle.com/javase/7/docs/technotes/guides/jvmti/)
+agent that needs to be enabled when JVM starts. The agent is enabled by
+using `-agentpath`
+[option](http://docs.oracle.com/javase/8/docs/platform/jvmti/jvmti.html#starting)
+with Java launcher. Most debugger options are configured through system
+properties.
+
+For example:
+
+<pre>
+java <b>-agentpath:/opt/cdbg/cdbg_java_agent.so</b> -jar ~/myapp.jar
+</pre>
+
+By default the debugger agent assumes that it runs on Google Compute Engine and
+uses local [metadata service](https://cloud.google.com/compute/docs/metadata) to
+obtain the credentials. You can still use the Java Cloud Debugger outside of
+Google Compute Engine or on a virtual machine that does not allow API access to
+all Google Cloud services. This would require setting up a
+[service account](#Service_Account).
+
+### Web Servers
+
+Java web servers usually start through a bootstrap process, and each web server
+has its own way of customizing Java options.
+
+#### Tomcat
+
+Add this line to `/etc/default/tomcat7` or `/etc/default/tomcat8`:
+
+```shell
+JAVA_OPTS="${JAVA_OPTS} -agentpath:/opt/cdbg/cdbg_java_agent.so"
+```
+
+If you run Tomcat in a Docker container, add this line to `Dockerfile` instead:
+
+```
+ENV JAVA_OPTS -agentpath:/opt/cdbg/cdbg_java_agent.so
+```
+
+#### Jetty
+
+Add `cdbg.ini` file to `/var/lib/jetty/start.d`:
+
+```ini
+--exec
+-agentpath:/opt/cdbg/cdbg_java_agent.so
+```
+
+### Versioning
+
+Developers normally run multiple versions of the application at the same time
+and multiple applications within the same project. You can tag each such entity
+with Cloud Debugger. This will make it easier to select the right application in
+the Cloud Debugger UI.
+
+To tag the version of the application, please add one or more of these optional
+system properties:
+
+<pre>
+-Dcom.google.cdbg.version=<i>myversion</i>
+-Dcom.google.cdbg.module=<i>mymodule</i>
+-Dcom.google.cdbg.minorversion=<i>minor version or build ID</i>
+</pre>
+
+### Logging
+
+By default the Java Cloud Debugger write its logs to `cdbg_java_agent.INFO` file
+in the default logging directory. It is possible to change the log directory
+as following:
+
+```
+-agentpath:/opt/cdbg/cdbg_java_agent.so=--log_dir=/my/log/dir
+```
+
+Alternatively you can make the Java Cloud Debugger log to *stderr*:
+
+```
+-agentpath:/opt/cdbg/cdbg_java_agent.so=--logtostderr=1
+```
+
+### Service Account
+
+Service account authentication lets you run the debugger agent on any Linux
+machine, including outside of [Google Cloud Platform](https://cloud.google.com).
+The debugger agent authenticates against the backend with the service account
+created in [Google Developers Console](https://console.developers.google.com).
+If your application runs on Google Compute Engine,
+[metadata service authentication](#Setup) is an easier option.
+
+The first step for this setup is to create the service account in .p12 format.
+Please see
+[OAuth](https://cloud.google.com/storage/docs/authentication?hl=en#generating-a-private-key)
+page for detailed instructions. If you don't have a Google Cloud Platform
+project, you can create one for free on
+[Google Developers Console](https://console.developers.google.com).
+
+Once you have the service account, please note the service account e-mail,
+[project ID and project number](https://developers.google.com/console/help/new/#projectnumber).
+Then copy the .p12 file to all the machines that run your application.
+
+You will need to install the debugger agent that supports the service account.
+The URL is: https://storage.googleapis.com/cloud-debugger/compute-java/debian-wheezy/cdbg_java_agent_service_account.tar.gz.
+
 
-For information on how to use the Cloud Debugger please see
-[Cloud Debugger](https://cloud.google.com/tools/cloud-debugger/).
+To enable the service account authentication add these arguments to Java
+launcher command (same way as with `-agentpath`):
 
-Google provides a binary distribution for Java Cloud Debugger that you can
-use. This open package lets you build it from source code instead.
+<pre>
+-Dcom.google.cdbg.auth.serviceaccount.enable=<i>true</i>
+-Dcom.google.cdbg.auth.serviceaccount.projectid=<i>myprojectid</i>
+-Dcom.google.cdbg.auth.serviceaccount.projectnumber=<i>123456789</i>
+-Dcom.google.cdbg.auth.serviceaccount.email=<i>email@something.com</i>
+-Dcom.google.cdbg.auth.serviceaccount.p12file=<i>/opt/cdbg/svc.p12</i>
+</pre>

src/agent/Makefile

@@ -140,6 +140,7 @@ HEADERS := \
 	$(wildcard *.h) \
 
 LIBS= \
+	-ldl \
 	-lrt \
 	-pthread \
 	$(THIRD_PARTY_LIB_PATH)/libjsoncpp.a \

src/agent/capture_data_collector.cc

@@ -25,13 +25,13 @@
 #include "method_locals.h"
 #include "model_util.h"
 #include "object_evaluator.h"
-#include "safe_method_caller.h"
 #include "value_formatter.h"
 
 namespace devtools {
 namespace cdbg {
 
-CaptureDataCollector::CaptureDataCollector() {
+CaptureDataCollector::CaptureDataCollector(JvmEvaluators* evaluators)
+    : evaluators_(evaluators) {
   // Reserve "var_table_index" 0 for memory objects that we didn't capture
   // because collector ran out of quota.
   memory_objects_.push_back(MemoryObject());
@@ -44,21 +44,14 @@ CaptureDataCollector::~CaptureDataCollector() {
 
 
 void CaptureDataCollector::Collect(
-    const Config& config,
-    ClassFilesCache* class_files_cache,
-    EvalCallStack* eval_call_stack,
-    ClassIndexer* class_indexer,
-    ClassMetadataReader* class_metadata_reader,
-    MethodLocals* method_locals,
-    ObjectEvaluator* object_evaluator,
     const std::vector<CompiledExpression>& watches,
     jthread thread) {
-  DCHECK(eval_call_stack_ == nullptr) << "Collect can only be called once";
-  eval_call_stack_ = CHECK_NOTNULL(eval_call_stack);
+  std::unique_ptr<MethodCaller> pretty_printers_method_caller =
+      evaluators_->method_caller_factory(Config::PRETTY_PRINTERS);
 
   // Walk the call stack.
   std::vector<EvalCallStack::JvmFrame> jvm_frames;
-  eval_call_stack_->Read(thread, &jvm_frames);
+  evaluators_->eval_call_stack->Read(thread, &jvm_frames);
 
   const int call_frames_count = jvm_frames.size();
   call_frames_.resize(call_frames_count);
@@ -68,22 +61,13 @@ void CaptureDataCollector::Collect(
     // Collect local variables.
     if ((depth < kMethodLocalsFrames) &&
         (jvm_frames[depth].code_location.method != nullptr)) {
-      // We don't expect any methods to be called while reading fields.
-      SafeMethodCaller method_caller(
-          &config,
-          Config::MethodCallQuota(),  // Default quota of zero for everything.
-          class_indexer,
-          class_files_cache);
-
       EvaluationContext evaluation_context;
       evaluation_context.thread = thread;
       evaluation_context.frame_depth = depth;
-      evaluation_context.method_caller = &method_caller;
+      evaluation_context.method_caller = pretty_printers_method_caller.get();
 
       ReadLocalVariables(
           evaluation_context,
-          class_metadata_reader,
-          method_locals,
           jvm_frames[depth].code_location.method,
           jvm_frames[depth].code_location.location,
           &call_frames_[depth].arguments,
@@ -107,20 +91,16 @@ void CaptureDataCollector::Collect(
       watch_results_[i].compile_error_message = {ExpressionSensitiveData, { }};
 
     } else if (watches[i].evaluator != nullptr) {
-      SafeMethodCaller method_caller(
-          &config,
-          config.GetQuota(Config::EXPRESSION_EVALUATION),
-          class_indexer,
-          class_files_cache);
+      std::unique_ptr<MethodCaller> expression_method_caller =
+          evaluators_->method_caller_factory(Config::EXPRESSION_EVALUATION);
 
       EvaluationContext evaluation_context;
       evaluation_context.thread = thread;
       evaluation_context.frame_depth = 0;
-      evaluation_context.method_caller = &method_caller;
+      evaluation_context.method_caller = expression_method_caller.get();
 
       EvaluateWatchedExpression(
           evaluation_context,
-          class_metadata_reader,
           *watches[i].evaluator,
           &watch_results_[i].evaluation_result);
 
@@ -142,16 +122,10 @@ void CaptureDataCollector::Collect(
   ++it_pending_object;  // First entry has a special meaning of "buffer full".
   ++captured_variable_table_size;
 
-  SafeMethodCaller method_caller(
-      &config,
-      config.GetQuota(Config::PRETTY_PRINTERS),
-      class_indexer,
-      class_files_cache);
-
   while ((it_pending_object != memory_objects_.end()) &&
          CanCollectMoreMemoryObjects()) {
-    object_evaluator->Evaluate(
-        &method_caller,
+    evaluators_->object_evaluator->Evaluate(
+        pretty_printers_method_caller.get(),
         it_pending_object->object_ref,
         &it_pending_object->members);
 
@@ -227,6 +201,11 @@ void CaptureDataCollector::Format(BreakpointModel* breakpoint) const {
       } else {
         object_variable.reset(new VariableModel);
 
+        object_variable->type = TypeNameFromSignature({
+            JType::Object,
+            GetObjectClassSignature(memory_object.object_ref)
+        });
+
         if (!memory_object.status.description.format.empty()) {
           object_variable->status =
               StatusMessageBuilder(memory_object.status).build();
@@ -245,8 +224,6 @@ void CaptureDataCollector::Format(BreakpointModel* breakpoint) const {
 
 void CaptureDataCollector::ReadLocalVariables(
     const EvaluationContext& evaluation_context,
-    ClassMetadataReader* class_metadata_reader,
-    MethodLocals* method_locals,
     jmethodID method,
     jlocation location,
     std::vector<NamedJVariant>* arguments,
@@ -257,7 +234,7 @@ void CaptureDataCollector::ReadLocalVariables(
   }
 
   std::shared_ptr<const MethodLocals::Entry> entry =
-      method_locals->GetLocalVariables(method);
+      evaluators_->method_locals->GetLocalVariables(method);
 
   // TODO(vlif): refactor this function to add locals and arguments to
   // output vectors as we go.
@@ -310,7 +287,6 @@ void CaptureDataCollector::ReadLocalVariables(
 
 void CaptureDataCollector::EvaluateWatchedExpression(
     const EvaluationContext& evaluation_context,
-    ClassMetadataReader* class_metadata_reader,
     const ExpressionEvaluator& watch_evaluator,
     NamedJVariant* result) {
   ErrorOr<JVariant> evaluation_result =
@@ -373,8 +349,9 @@ std::unique_ptr<VariableModel> CaptureDataCollector::FormatVariable(
       ValueFormatter::Format(
           source,
           ValueFormatter::Options(),
-          &formatted_value);
-      target->value = formatted_value;
+          &formatted_value,
+          &target->type);
+      target->value = std::move(formatted_value);
     } else {
       jobject ref = nullptr;
       const int* var_table_index = nullptr;
@@ -408,7 +385,7 @@ std::unique_ptr<VariableModel> CaptureDataCollector::FormatVariable(
 string CaptureDataCollector::GetFunctionName(int depth) const {
   const int frame_info_key = call_frames_[depth].frame_info_key;
   const auto& frame_info =
-      eval_call_stack_->ResolveCallFrameKey(frame_info_key);
+      evaluators_->eval_call_stack->ResolveCallFrameKey(frame_info_key);
 
   string function_name =
       TypeNameFromJObjectSignature(frame_info.class_signature);
@@ -423,7 +400,7 @@ std::unique_ptr<SourceLocationModel>
 CaptureDataCollector::GetCallFrameSourceLocation(int depth) const {
   const int frame_info_key = call_frames_[depth].frame_info_key;
   const auto& frame_info =
-      eval_call_stack_->ResolveCallFrameKey(frame_info_key);
+      evaluators_->eval_call_stack->ResolveCallFrameKey(frame_info_key);
 
   std::unique_ptr<SourceLocationModel> location(new SourceLocationModel);