spark-excel icon indicating copy to clipboard operation
spark-excel copied to clipboard

Co-maintainers wanted

Open nightscape opened this issue 5 years ago β€’ 25 comments

Due to personal and professional constraints, the development of this library has been rather slow. If you find value in this library, please consider stepping up as a co-maintainer. Help is very welcome e.g. in the following areas:

  • Additional features
  • Code improvements and reviews
  • Bug analysis and fixing
  • Documentation improvements
  • Build / test infrastructure

If you think you can contribute in any of the above areas, please leave a comment below and I will get in touch with you on how to best proceed.

Thank you!

nightscape avatar Jan 06 '20 21:01 nightscape

I would offer to help with maintaining this package, but I have to confess I haven’t done any Scala development in the past so at most I could probably help with grooming issues. I was able to get the code to build without problems but I was never able to get the jar to work on Databricks (even the master branch).

From: Martin Mauch [email protected] Reply-To: crealytics/spark-excel [email protected] Date: Monday, January 6, 2020 at 1:35 PM To: crealytics/spark-excel [email protected] Cc: Subscribed [email protected] Subject: [crealytics/spark-excel] Co-maintainers wanted (#191)

CAUTION: This email originated from outside of Thermo Fisher Scientific. If you believe it to be suspicious, report using the Report Phish button in Outlook or send to [email protected].

Due to personal and professional constraints, the development of this library has been rather slow. If you find value in this library, please consider stepping up as a co-maintainer. Help is very welcome e.g. in the following areas:

  • Additional features
  • Code improvements and reviews
  • Bug analysis and fixing
  • Documentation improvements
  • Build / test infrastructure

If you think you can contribute in any of the above areas, please leave a comment below and I will get in touch with you on how to best proceed.

Thank you!

β€” You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_crealytics_spark-2Dexcel_issues_191-3Femail-5Fsource-3Dnotifications-26email-5Ftoken-3DAM2MKX6VJ2CV6XFEZ26NEKTQ4OPZJA5CNFSM4KDLKHS2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4IEKSB3A&d=DwMCaQ&c=q6k2DsTcEGCcCb_WtVSz6hhIl8hvYssy7sH8ZwfbbKU&r=1AYsaX68NpPh6fU1okBqmHQAQzvdEdHjWewZcHC-lL4&m=h7QfRVbLhq2U0hRirL_SkJ7pvy61hBIdqxzDiH5cdfw&s=DS4d-YiGMLk7wL5cp7XXWbfeI9kA2p0BghySN3ODXM8&e=, or unsubscribehttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AM2MKX6CP2K5SPIHUCOANZTQ4OPZJANCNFSM4KDLKHSQ&d=DwMCaQ&c=q6k2DsTcEGCcCb_WtVSz6hhIl8hvYssy7sH8ZwfbbKU&r=1AYsaX68NpPh6fU1okBqmHQAQzvdEdHjWewZcHC-lL4&m=h7QfRVbLhq2U0hRirL_SkJ7pvy61hBIdqxzDiH5cdfw&s=oN3DBPlA8htOsyR8R-5780c465nYLIZjG5QKOAlp47Q&e=.

carlo-quinonez avatar Jan 07 '20 00:01 carlo-quinonez

Hi @carlo-quinonez, thank you! Issue grooming would be quite helpful. Regarding your trouble with Databricks: Is there already an existing issue that describes the problem? If so, you could mention me in a comment there and I'll try to help. Otherwise please open a new one.

nightscape avatar Jan 09 '20 08:01 nightscape

Hi @nightscape - @carlo-quinonez is my corporate account and my company wants us to use personal accounts for working on public projects.

luzlab avatar Jan 10 '20 23:01 luzlab

Can you make a collaborator so I can tag issues?

luzlab avatar Jan 11 '20 00:01 luzlab

Hello @nightscape! I am trying to write multiple dataframes into an excel file which will contain multiple sheets inside it.

How do I provide a name for those individual excel sheets?

Here's what I am trying to do: df1.write().format("com.crealytics.spark.excel").option("sheetName","My Sheet 1").option("header", "true").save("hdfs://127.0.0.1:9000/var/www/" + outFile + ".xls");

df2.write().format("com.crealytics.spark.excel").option("sheetName","My Sheet 2").option("header", "true").mode(SaveMode.Append).save("hdfs://127.0.0.1:9000/var/www/" + outFile + ".xls");

NiranjanBalasubramani avatar Feb 28 '20 07:02 NiranjanBalasubramani

Hey @NiranjanBalasubramani, wrong place πŸ˜‰ This issue is about contributing to the project. Questions are probably best asked on StackOverflow.

nightscape avatar Feb 28 '20 11:02 nightscape

@nightscape Done πŸ˜ƒ https://stackoverflow.com/questions/60500266/how-to-mention-individual-sheet-names-while-writing-mutiple-org-apache-spark-sql

NiranjanBalasubramani avatar Mar 03 '20 05:03 NiranjanBalasubramani

Hey! I would like to be a co-maintener. How can I help? I have large experience on Scala language and other JVM languages.

daviddelucca avatar Mar 09 '20 11:03 daviddelucca

Hi @daviddelucca, great! I've added you as a collaborator πŸ˜ƒ πŸ‘

nightscape avatar Mar 09 '20 17:03 nightscape

Hi @nightscape, I would like to contribute to this project. I already have a fix for the https://github.com/crealytics/spark-excel/issues/327 issue which is working for me in local. I also made a local working copy of spark csv parser like columnNameOfCorruptRecord option to this project .

Harivignesh3419 avatar Dec 30 '20 15:12 Harivignesh3419

Hey @Harivignesh3419 sounds great! Can you open a PR for it? I'll add you as a contributor after your first successful PR :+1:

nightscape avatar Dec 30 '20 21:12 nightscape

Hi @nightscape , I would like to contribute to this project. Areas those I will be able to contribute:

  • Fixing these two issue on input_file_name() and loading multiple excel files with glob pattern #52 #74 #97
  • Upgrade to spark data source API 2.0
  • And would like to simplify the project dependencies by dropping the writing support (we can discuss the detail later)

Thanks

quanghgx avatar May 02 '21 08:05 quanghgx

Hi @quanghgx, great!! For the first two points it would probably be good to start with the open PR to make Excel a FileFormat. That would have to be rebased on latest master branch first though. Afair the existing tests were already working, but I added new tests for reading and writing multiple files and I think those failed.

nightscape avatar May 02 '21 12:05 nightscape

Thanks @nightscape that's great. I haven't known this branch before. Detail will be on the PR thread. Let me some time.

quanghgx avatar May 02 '21 14:05 quanghgx

Maybe I can help with the documentation, I am new to Spark and Scala.

rudolph-m avatar Nov 26 '21 14:11 rudolph-m

@rudolph-m that would be great! Being new to sth. leaves no room for assumptions about implicit knowledge.

nightscape avatar Nov 26 '21 15:11 nightscape

let me know how I can assist. I found this repo while I was looking for a way to read excel files in Spark and it was not easy for me to understand how to use the library

rudolph-m avatar Nov 30 '21 06:11 rudolph-m

@rudolph-m could you maybe open a new issue regarding improving the documentation and mention there what you found hard to discover and what information would have helped you? We could provide you with the information then and you expand it into sth. understandable. WDYT?

nightscape avatar Nov 30 '21 14:11 nightscape

Hello, I would like to contribute to the development and maintenance of this repo. Pls let me know how and where can I be of help.

SubhasisMukherjee avatar Dec 09 '21 20:12 SubhasisMukherjee

Hey @SubhasisMukherjee, great!! Do you have some Scala experience, or mostly Python?

nightscape avatar Dec 09 '21 21:12 nightscape

Hey @nightscape I primarily work with Python and PySpark. I have learnt Scala as well recently, but not having project experience on Scala, though I am comfortable with reading and modifying codes in Scala.

SubhasisMukherjee avatar Dec 10 '21 02:12 SubhasisMukherjee

@SubhasisMukherjee hope you had nice holidays and a good start into the new year πŸ˜ƒ Sorry for not getting back earlier... Do you happen to use spark-excel in some cloud environment like Azure, AWS or GCS?

nightscape avatar Jan 17 '22 17:01 nightscape

@nightscape I can go through some of the issues and see if they can be quickly resolved. If I get some more time, I don't mind helping to maintain this project. Let me know if you need anything from me.

williamdphillips avatar Apr 04 '23 15:04 williamdphillips

@williamdphillips great! I added you to the project members πŸ‘

nightscape avatar Apr 04 '23 17:04 nightscape

hi, I can take complete ownership of the maintenance of the package. Let me know if you have got any responses for this co-mainteners.

Balachandar-Ganesan avatar Sep 24 '23 08:09 Balachandar-Ganesan