Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,5 +16,5 @@
blank_issues_enabled: false
contact_links:
- name: Gluten Discussions
url: https://github.com/apache/incubator-gluten/discussions
url: https://github.com/apache/gluten/discussions
about: Ask questions or discuss new feature ideas here.
2 changes: 1 addition & 1 deletion .github/PULL_REQUEST_TEMPLATE
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
Thank you for submitting a pull request! Here are some tips:

1. For first-time contributors, please read our contributing guide:
https://github.com/apache/incubator-gluten/blob/main/CONTRIBUTING.md
https://github.com/apache/gluten/blob/main/CONTRIBUTING.md
2. If necessary, create a GitHub issue for discussion beforehand to avoid duplicate work.
3. If the PR is specific to a single backend, include [VL] or [CH] in the PR title to indicate the
Velox or ClickHouse backend, respectively.
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/dev_cron/pr_issue_linker.js
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ function detectIssueID(title) {
}

async function appendToPRDescription(github, context, pullRequestNumber, issuesID) {
const issueURL = `https://github.com/apache/incubator-gluten/issues/${issuesID}`;
const issueURL = `https://github.com/apache/gluten/issues/${issuesID}`;
const issueReference = `#${issuesID}`

// Fetch the current PR description.
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/velox_weekly.yml
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ jobs:
export PATH=$JAVA_HOME/bin:$PATH

# action/checkout does not work centos7 anymore, so we clone the branch instead.
git clone -b main --depth=1 https://github.com/apache/incubator-gluten.git && cd incubator-gluten/
git clone -b main --depth=1 https://github.com/apache/gluten.git && cd gluten/
if [ ${{ github.event_name }} = "pull_request" ]; then
git fetch origin ${{ github.ref }}:pr_branch && git checkout pr_branch
fi
Expand Down
2 changes: 1 addition & 1 deletion .idea/vcs.xml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

10 changes: 5 additions & 5 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,15 +44,15 @@ please add at least one UT to ensure code quality and reduce regression issues f

Please update document for your proposed code change if necessary.

If a new config property is being introduced, please update [Configuration.md](https://github.com/apache/incubator-gluten/blob/main/docs/Configuration.md).
If a new config property is being introduced, please update [Configuration.md](https://github.com/apache/gluten/blob/main/docs/Configuration.md).

### Code Style

##### Java/Scala code style
Developer can import the code style setting to IDE and format Java/Scala code with spotless maven plugin. See [Java/Scala code style](https://github.com/apache/incubator-gluten/blob/main/docs/developers/NewToGluten.md#javascala-code-style).
Developer can import the code style setting to IDE and format Java/Scala code with spotless maven plugin. See [Java/Scala code style](https://github.com/apache/gluten/blob/main/docs/developers/NewToGluten.md#javascala-code-style).

##### C/C++ code style
There are some code style conventions need to comply. See [CppCodingStyle.md](https://github.com/apache/incubator-gluten/blob/main/docs/developers/CppCodingStyle.md).
There are some code style conventions need to comply. See [CppCodingStyle.md](https://github.com/apache/gluten/blob/main/docs/developers/CppCodingStyle.md).

For Velox backend, developer can just execute `dev/formatcppcode.sh` to format C/C++ code. It requires `clang-format-15`
installed in your development env.
Expand All @@ -68,7 +68,7 @@ You can execute a script to fix license header issue, as the following shows.
### Gluten CI

##### ClickHouse Backend CI
To check CI failure for CH backend, please log in with the public account/password provided [here](https://github.com/apache/incubator-gluten/blob/main/docs/get-started/ClickHouse.md#new-ci-system).
To check CI failure for CH backend, please log in with the public account/password provided [here](https://github.com/apache/gluten/blob/main/docs/get-started/ClickHouse.md#new-ci-system).
To re-trigger CH CI, please post the below comment on PR page:

`Run Gluten Clickhouse CI`
Expand All @@ -79,7 +79,7 @@ To check CI failure for Velox backend, please go into the GitHub action page fro
To see the perf. impact on Velox backend, you can comment `/Benchmark Velox` on PR page to trigger a pretest. The benchmark
(currently TPC-H) result will be posted after completed.

If some new dependency is required to be installed, you may need to do some change for CI docker at [this folder](https://github.com/apache/incubator-gluten/tree/main/tools/gluten-te).
If some new dependency is required to be installed, you may need to do some change for CI docker at [this folder](https://github.com/apache/gluten/tree/main/tools/gluten-te).

### Code Review

Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,7 @@ to view past discussions, or [subscribe to the mailing list](mailto:dev-subscrib

### Slack Channel (English)

Request an invitation to the ASF Slack workspace via [this page](https://github.com/apache/incubator-gluten/discussions/8429). Once invited, you can join the **incubator-gluten** channel.
Request an invitation to the ASF Slack workspace via [this page](https://github.com/apache/gluten/discussions/8429). Once invited, you can join the **gluten** channel.

The ASF Slack login entry: https://the-asf.slack.com/.

Expand Down Expand Up @@ -166,6 +166,6 @@ Gluten is licensed under [Apache License Version 2.0](https://www.apache.org/lic

Gluten was initiated by Intel and Kyligence in 2022. Several other companies are also actively contributing to its development, including BIGO, Meituan, Alibaba Cloud, NetEase, Baidu, Microsoft, IBM, Google, etc.

<a href="https://github.com/apache/incubator-gluten/graphs/contributors">
<img src="https://contrib.rocks/image?repo=apache/incubator-gluten&columns=25" />
<a href="https://github.com/apache/gluten/graphs/contributors">
<img src="https://contrib.rocks/image?repo=apache/gluten&columns=25" />
</a>
Original file line number Diff line number Diff line change
Expand Up @@ -181,7 +181,7 @@ object CHRuleApi {
}

/**
* Since https://github.com/apache/incubator-gluten/pull/883.
* Since https://github.com/apache/gluten/pull/883.
*
* TODO: Remove this since tricky to maintain.
*/
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ object CHValidatorApi {
* and make it into a new column which the shuffle will refer to. But we need to remove it from
* the result columns from the shuffle.
*
* Since https://github.com/apache/incubator-gluten/pull/1071.
* Since https://github.com/apache/gluten/pull/1071.
*/
def supportShuffleWithProject(outputPartitioning: Partitioning, child: SparkPlan): Boolean = {
child match {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ case class CHColumnarToCarrierRowExec(override val child: SparkPlan)
override def rowType0(): Convention.RowType = CHCarrierRowType
override protected def withNewChildInternal(newChild: SparkPlan): SparkPlan =
copy(child = newChild)
// Since https://github.com/apache/incubator-gluten/pull/1595.
// Since https://github.com/apache/gluten/pull/1595.
override protected def doExecuteColumnar(): RDD[ColumnarBatch] = {
if (child.supportsColumnar) {
child.executeColumnar()
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ import org.apache.spark.sql.catalyst.expressions.aggregate.AggregateFunction

// Static helper object for handling expressions that are specifically used in CH backend.
object CHExpressions {
// Since https://github.com/apache/incubator-gluten/pull/1937.
// Since https://github.com/apache/gluten/pull/1937.
def createAggregateFunction(context: SubstraitContext, aggregateFunc: AggregateFunction): Long = {
val expressionExtensionTransformer =
ExpressionExtensionTrait.findExpressionExtension(aggregateFunc.getClass)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -861,7 +861,7 @@ class GlutenFunctionValidateSuite extends GlutenClickHouseWholeStageTransformerS
}

test("test parse string with blank to integer") {
// issue https://github.com/apache/incubator-gluten/issues/4956
// issue https://github.com/apache/gluten/issues/4956
val sql = "select cast(concat(' ', cast(id as string)) as bigint) from range(10)"
runQueryAndCompare(sql)(checkGlutenPlan[ProjectExecTransformer])
}
Expand Down Expand Up @@ -915,7 +915,7 @@ class GlutenFunctionValidateSuite extends GlutenClickHouseWholeStageTransformerS
}
}

test("test issue: https://github.com/apache/incubator-gluten/issues/6561") {
test("test issue: https://github.com/apache/gluten/issues/6561") {
val sql =
"""
|select
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ class GlutenClickhouseFunctionSuite extends ParquetSuite {
}
}

test("https://github.com/apache/incubator-gluten/issues/6938") {
test("https://github.com/apache/gluten/issues/6938") {
val testSQL =
s"""
|select * from (
Expand Down Expand Up @@ -372,7 +372,7 @@ class GlutenClickhouseFunctionSuite extends ParquetSuite {
}
}

test("GLUTEN-7545: https://github.com/apache/incubator-gluten/issues/7545") {
test("GLUTEN-7545: https://github.com/apache/gluten/issues/7545") {
withTable("regexp_test") {
sql("create table if not exists regexp_test (id string) using parquet")
sql("insert into regexp_test values('1999-6-1')")
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -514,7 +514,7 @@ class GlutenClickHouseNativeWriteTableSuite
("decimal_field", "decimal(23,12)"),
("date_field", "date")
// ("timestamp_field", "timestamp")
// FIXME https://github.com/apache/incubator-gluten/issues/8053
// FIXME https://github.com/apache/gluten/issues/8053
)
val origin_table = "origin_table"
withSource(genTestData(), origin_table) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -319,7 +319,7 @@ class GlutenClickHouseTPCHColumnarShuffleParquetAQESuite extends ParquetTPCHSuit
}
}

ignore("https://github.com/apache/incubator-gluten/issues/7726") {
ignore("https://github.com/apache/gluten/issues/7726") {
runQueryAndCompare(Arm.withResource(
Source.fromFile(new File(s"$queryPath/tpch-schema-related/7726.sql"), "UTF-8"))(_.mkString)) {
df =>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -325,7 +325,7 @@ class VeloxIcebergSuite extends IcebergSuite {
val lastExecId = statusStore.executionsList().last.executionId
val executionMetrics = statusStore.executionMetrics(lastExecId)

// TODO: fix https://github.com/apache/incubator-gluten/issues/11510
// TODO: fix https://github.com/apache/gluten/issues/11510
assert(executionMetrics(metrics("numWrittenFiles").id).toLong == 0)
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,7 @@ object VeloxBackendSettings extends BackendSettingsApi {
case ParquetReadFormat =>
val parquetOptions = new ParquetOptions(CaseInsensitiveMap(properties), SQLConf.get)
if (parquetOptions.mergeSchema) {
// https://github.com/apache/incubator-gluten/issues/7174
// https://github.com/apache/gluten/issues/7174
Some(s"not support when merge schema is true")
} else {
None
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -315,7 +315,7 @@ object VeloxListenerApi {

var parsed: Map[String, String] = GlutenConfigUtil.parseConfig(conf.getAll.toMap)

// Workaround for https://github.com/apache/incubator-gluten/issues/7837
// Workaround for https://github.com/apache/gluten/issues/7837
if (isDriver && !inLocalMode(conf)) {
parsed += (COLUMNAR_VELOX_CACHE_ENABLED.key -> "false")
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -90,11 +90,11 @@ class VeloxValidatorApi extends ValidatorApi {
child: SparkPlan): Option[String] = {
if (!BackendsApiManager.getSettings.supportEmptySchemaColumnarShuffle()) {
if (outputAttributes.isEmpty) {
// See: https://github.com/apache/incubator-gluten/issues/7600.
// See: https://github.com/apache/gluten/issues/7600.
return Some("Shuffle with empty output schema is not supported")
}
if (child.output.isEmpty) {
// See: https://github.com/apache/incubator-gluten/issues/7600.
// See: https://github.com/apache/gluten/issues/7600.
return Some("Shuffle with empty input schema is not supported")
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ class ArrowCsvScanWithTableCacheSuite extends ArrowCsvScanSuiteBase {
}

/**
* Test for GLUTEN-8453: https://github.com/apache/incubator-gluten/issues/8453. To make sure no
* Test for GLUTEN-8453: https://github.com/apache/gluten/issues/8453. To make sure no
* error is thrown when caching an Arrow Java query plan.
*/
test("csv scan v1 with table cache") {
Expand All @@ -107,7 +107,7 @@ class ArrowCsvScanWithTableCacheSuite extends ArrowCsvScanSuiteBase {
}
}

/** Since https://github.com/apache/incubator-gluten/pull/5850. */
/** Since https://github.com/apache/gluten/pull/5850. */
@Ignore
abstract class ArrowCsvScanSuite extends ArrowCsvScanSuiteBase {

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1998,7 +1998,7 @@ class MiscOperatorSuite extends VeloxWholeStageTransformerSuite with AdaptiveSpa
}
}

// Enable the test after fixing https://github.com/apache/incubator-gluten/issues/6827
// Enable the test after fixing https://github.com/apache/gluten/issues/6827
ignore("Test round expression") {
val df1 = runQueryAndCompare("SELECT round(cast(0.5549999999999999 as double), 2)") { _ => }
checkLengthAndPlan(df1, 1)
Expand Down Expand Up @@ -2032,7 +2032,7 @@ class MiscOperatorSuite extends VeloxWholeStageTransformerSuite with AdaptiveSpa
}
}

// Since https://github.com/apache/incubator-gluten/pull/7330.
// Since https://github.com/apache/gluten/pull/7330.
test("field names contain non-ASCII characters") {
withTempPath {
path =>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ class VeloxColumnarCacheSuite extends VeloxWholeStageTransformerSuite with Adapt
}
}

// See issue https://github.com/apache/incubator-gluten/issues/8497.
// See issue https://github.com/apache/gluten/issues/8497.
test("Input fallen back vanilla Spark columnar scan") {
def withId(id: Int): Metadata =
new MetadataBuilder().putLong("parquet.field.id", id).build()
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ class VeloxTransitionSuite extends SharedSparkSession with TransitionSuiteBase {
val out = BackendTransitions.insert(in, outputsColumnar = false)
// No explicit transition needed for ArrowNative-to-Velox.
// FIXME: Add explicit transitions.
// See https://github.com/apache/incubator-gluten/issues/7313.
// See https://github.com/apache/gluten/issues/7313.
assert(
out == VeloxColumnarToRowExec(
BatchUnary(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -496,7 +496,7 @@ abstract class ScalarFunctionsValidateSuite extends FunctionsValidateSuite {
}
}

// FIXME: Ignored: https://github.com/apache/incubator-gluten/issues/7600.
// FIXME: Ignored: https://github.com/apache/gluten/issues/7600.
ignore("monotonically_increasintestg_id") {
runQueryAndCompare("""SELECT monotonically_increasing_id(), l_orderkey
| from lineitem limit 100""".stripMargin) {
Expand Down
4 changes: 2 additions & 2 deletions cpp-ch/local-engine/Parser/ExpressionParser.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -345,7 +345,7 @@ ExpressionParser::NodeRawConstPtr ExpressionParser::parseExpression(ActionsDAG &
}
else if ((isMap(denull_input_type) || isArray(denull_input_type) || isTuple(denull_input_type)) && isString(denull_output_type))
{
/// https://github.com/apache/incubator-gluten/issues/9049
/// https://github.com/apache/gluten/issues/9049
result_node = toFunctionNode(actions_dag, "sparkCastComplexTypesToString", args);
}
else if (isString(denull_input_type) && substrait_type.has_bool_())
Expand All @@ -357,7 +357,7 @@ ExpressionParser::NodeRawConstPtr ExpressionParser::parseExpression(ActionsDAG &
else if (isString(denull_input_type) && isInt(denull_output_type))
{
/// Spark cast(x as INT) if x is String -> CH cast(trim(x) as INT)
/// Refer to https://github.com/apache/incubator-gluten/issues/4956 and https://github.com/apache/incubator-gluten/issues/8598
/// Refer to https://github.com/apache/gluten/issues/4956 and https://github.com/apache/gluten/issues/8598
const auto * trim_str_arg = addConstColumn(actions_dag, std::make_shared<DataTypeString>(), " \t\n\r\f");
args[0] = toFunctionNode(actions_dag, "trimBothSpark", {args[0], trim_str_arg});
args.emplace_back(addConstColumn(actions_dag, std::make_shared<DataTypeString>(), output_type->getName()));
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ class FunctionParserArrayTransform : public FunctionParser
{
/// Convert Array(T) to Array(U) if needed, Array(T) is the type of the first argument of transform.
/// U is the argument type of lambda function. In some cases Array(T) is not equal to Array(U).
/// e.g. in the second query of https://github.com/apache/incubator-gluten/issues/6561, T is String, and U is Nullable(String)
/// e.g. in the second query of https://github.com/apache/gluten/issues/6561, T is String, and U is Nullable(String)
/// The difference of both types will result in runtime exceptions in function capture.
const auto & src_array_type = parsed_args[0]->result_type;
DataTypePtr dst_array_type = std::make_shared<DataTypeArray>(lambda_args.front().type);
Expand Down
4 changes: 2 additions & 2 deletions cpp-ch/local-engine/Storages/Output/NormalFileWriter.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ const std::vector<std::string> FileNameGenerator::SUPPORT_PLACEHOLDERS{"{id}", "

/// For Nullable(Map(K, V)) or Nullable(Array(T)), if the i-th row is null, we must make sure its nested data is empty.
/// It is for ORC/Parquet writing compatiability. For more details, refer to
/// https://github.com/apache/incubator-gluten/issues/8022 and https://github.com/apache/incubator-gluten/issues/8021
/// https://github.com/apache/gluten/issues/8022 and https://github.com/apache/gluten/issues/8021
static ColumnPtr truncateNestedDataIfNull(const ColumnPtr & column)
{
if (const auto * col_const = checkAndGetColumn<ColumnConst>(column.get()))
Expand Down Expand Up @@ -166,7 +166,7 @@ DB::Block NormalFileWriter::castBlock(const DB::Block & block) const

/// In case input block didn't have the same types as the preferred schema, we cast the input block to the preferred schema.
/// Notice that preferred_schema is the actual file schema, which is also the data schema of current inserted table.
/// Refer to issue: https://github.com/apache/incubator-gluten/issues/6588
/// Refer to issue: https://github.com/apache/gluten/issues/6588
size_t index = 0;
const auto & preferred_schema = file->getPreferredSchema();
for (auto & column : res)
Expand Down
2 changes: 1 addition & 1 deletion cpp/velox/operators/plannodes/CudfVectorStream.cc
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ bool CudfVectorStreamBase::hasNext() {
// driver to make the current task open to spilling.
//
// When a task is getting spilled, it should have been suspended so has zero running threads, otherwise there's
// possibility that this spill call hangs. See https://github.com/apache/incubator-gluten/issues/7243.
// possibility that this spill call hangs. See https://github.com/apache/gluten/issues/7243.
// As of now, non-zero running threads usually happens when:
// 1. Task A spills task B;
// 2. Task A tries to grow buffers created by task B, during which spill is requested on task A again.
Expand Down
2 changes: 1 addition & 1 deletion cpp/velox/operators/plannodes/RowVectorStream.cc
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ bool RowVectorStream::hasNext() {
// driver to make the current task open to spilling.
//
// When a task is getting spilled, it should have been suspended so has zero running threads, otherwise there's
// possibility that this spill call hangs. See https://github.com/apache/incubator-gluten/issues/7243.
// possibility that this spill call hangs. See https://github.com/apache/gluten/issues/7243.
// As of now, non-zero running threads usually happens when:
// 1. Task A spills task B;
// 2. Task A tries to grow buffers created by task B, during which spill is requested on task A again.
Expand Down
Loading
Loading